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PREFACE 


The Board of Education of the Los Angeles Unified School District voted in 
mid-1975 to contract with an outside group to study gains in 6th grade reading 
achievement in selected elementary schools with predominantly minority popula¬ 
tions. The impetus for the study was the installation of the School Preferred Reading 
Program in 1972. In January 1976 the board awarded a contract to Rand, calling 
for an analysis of the program’s effects on reading progress for Black and Mexican 
American students. 
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SUMMARY 


Increasing students’ reading achievement, particularly among minority stu¬ 
dents, has long been a central concern of the Los Angeles Unified School District. 
To further that goal, the LAUSD instituted its School Preferred Reading Program 
in 1972. In 1975, it contracted with The Rand Corporation to conduct the present 
study, whose purpose is to identify the school and classroom policies and other 
factors that have been most successful in raising the reading scores of inner-city 
children. 

We selected a sample of 20 elementary schools for analysis. All displayed large 
or consistent gains (in percentile points on national norms) for the 6th grade on the 
CTBS reading examination between 1972 and 1975, had predominantly minority- 
group student bodies (about half were mostly Mexican American, the other half 
mostly Black), were located in low-income neighborhoods, and had 1975 enrollments 
of at least 400 students. 

All principals and reading specialists present in 1976 were personally inter¬ 
viewed, and a questionnaire was administered to 81 of 83 teachers who taught 6th 
grade in these schools in 1974 and 1975. The purpose was to gather information on 
school leadership, reading program content and implementation, classroom atmos¬ 
phere, and teacher attributes. 

Background and demographic information, and reading test scores (from grades 
3 through 6), were recorded for individual members of the 6th grade classes of 1974 
and 1975 in 32 district junior high schools. The students’ 6th grade classroom and 
school experiences, as identified in the elementary school data collection, were 
linked to their test scores and background information to construct a longitudinal 
file. Students for whom it proved possible to collect such data were found to be amply 
representative of students who graduated from the 20 schools in the sample in 1974 
and 1975. Our analyses were conducted on these longitudinal files, not on compari¬ 
sons of scores of successive 6th grade classes in a given school—the method the 
district typically uses to measure reading progress. The latter can easily be distorted 
by changes in student body characteristics from one year to the next. Although the 
schools in our sample gained appreciably in reading achievement between 1974 and 
1975, the average 1975 6th grader in our sample declined slightly on national norms 
from his or her 5th grade rank. 

The tests used by the district were found to be reliable, as technically defined, 
according to standards of internal consistency and parallel form. High reliability 
was found not only for the sample as a whole but for individual schools. 

The validity of a few of the reading gains registered in sample 6th grade class¬ 
rooms was thrown into doubt by interview reports of test exposure. These reports 
were matched by patterns of unusually high gains for large fractions of students in 
these same classrooms (a total of six out of 78). The data from these classrooms were 
excluded from the analysis. In four additional classrooms, gains were as large and 
as concentrated, but we received no field reports of excessive test preparation. We 
performed our analysis with these four classrooms included, and then repeated it 
after deleting them, to measure the sensitivity of our results to varying judgments 
as to the extent of test exposure. 
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Background factors (such as socioeconomic status, health, ethnicity, and attend¬ 
ance) and reading test scores in earlier grades were found to account for the largest 
part of the variation in 6th grade reading scores for the children in our sample. For 
both Black and Mexican American children, however, other variables reflecting 
particular school and classroom experiences also had a significant influence. In 
contrast to some previous reseai'ch, we found that the school, the teacher, and 
combinations of school and teacher did make a difference in explaining measured 
reading proficiency. We also found that principals in both ethnic subsamples were 
able, with surprising accuracy, to identify and rank teachers in their schools by the 
teachers’ ability to bring about reading progress (as measured by data on individual 
student gains, which were not available to the principals when they assessed the 
teachers). 

In analyzing specific factors associated with observed gains in reading perfor¬ 
mance, we obtained strong results only for the sample of Black students. For them, 
the following factors were significant: 

. Teacher training in the use of a variety of materials keyed to individual 
student needs 

* Teachers who felt efficacious 

• Maintenance of orderly classrooms 

* High levels of parent-teacher contact 

« Teacher flexibility in modifying and adapting instructional approaches 

• Frequent informal consultations among teachers in implementing reading 
programs 

The first three factors were significant even when our analysis excluded all class¬ 
rooms in which the validity of test outcomes was uncertain. Confidence in the 
findings with respect to the last three factors was lessened, however, when we 
adopted this maximally skeptical procedure. 

We also conducted a supplementary study—largely subjective, because of limit¬ 
ed resources—of community involvement in the schools. We concluded that, in 
Black neighborhoods, the more vigorous were the schools’ efforts to involve parents 
and community in school decisionmaking, the better did 6th grade students in those 
schools fare in reading attainment. 

Factors not associated statistically with the reading gains of Black children 
included specific reading program or system, reading strategy (e.g., phonics versus 
comprehension), and such teacher attributes as ethnicity, experience, and type of 
education. 

Our inability to identify specific factors associated with the reading gains of 
Mexican American students—which in fact were almost identical overall to the 
gains registered by Black students—was probably due to failure to include or accu¬ 
rately measure some key variables. For example, we had no data on variations in 
English fluency across the sample of Mexican American children. 

Our findings support the continuation of the LAUSD’s School Preferred Reading 
Program, which features school autonomy and teacher flexibility. Because princi¬ 
pals seem able to identify effective practices, because teachers need the freedom to 
adapt and the training for individualization, and because no prefabricated reading 
system or strategy can provide a ’'quick fix” to the complex problem of reading 
instruction, a good measure of decisionmaking authority at the local school level 
seems warranted. 
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Chapter 1 

INTRODUCTION 


The impetus for this study arose as the Los Angeles Unified School District 
(LAUSDJ was about to complete its third year under the School Preferred Reading 
Program. The program concentrated on elementary students who were scoring at 
the 25th percentile or lower on standardized reading tests, it featured determination 
of goals and approaches by individual schools, and it set minimum quantitative 
objectives for reading gains at the 3d and 6th grade levels. The first year of the 
program, 1972-73, emphasized diagnosis and program design; the following two 
years incorporated the devised reforms into ongoing school operations. 

The purpose of the study was to identify those aspects of school reading pro¬ 
grams that were associated with substantial and consistent gains in standardized 
reading test scores among minority children. 

The terms of the contract award to Rand specified that the outcomes in reading 
achievement were to be gauged in terms of scores on standardized tests. Many 
analysts of the educational process—including some members of the Rand study 
team—have expressed doubts about the accuracy of such tests as indicators of 
students’educational progress, particularly for students from minority backgrounds. 
Because test scores are recognized and accepted by many others, however, we felt 
it important and worthwhile to try to discover which classroom approaches and 
which school policies were associated with increases in such scores. 

As part of this effort we sought to determine whether the observed gains were 
valid and reliable, to identify school and program characteristics that accounted for 
the valid gains, and to provide advice on the applicability of successful programs to 
other schools and to future school operations. 

Rand conceived this task as the identification of school and district policies that 
hold the most promise for improving reading achievement in minority schools. 
Therefore, the study does not focus on the teacher-student interactions that are 
often the basis for "process” analyses of education, such as how directive teachers 
are, how clear their presentations are, how frequently they ask questions, or how 
they reward students for good work. Instead, our analysis concentrates on how 
policy variables—for example, the content of schools’ reading programs, the ways 
the programs are implemented, and support from principals—are associated with 
gains and losses in achievement test scores. These kinds of variables are only a part 
of what matters in the schooling process, but we believe they represent the bulk of 
the factors that can be directly measured and influenced by people involved in school 
policy decisions: administrators, teacher groups, and parents. Consequently, our 
policy analysis seeks to identify those reading programs and school and classroom 
environmental factors that can be most readily altered to improve students’ reading 
skills. 
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RESEARCH APPROACH 

We became convinced early in the study that the criterion of improvement must 
be the gains in reading achievement over time for a given group of students as they 
progress from grade to grade. Year-to-year gains at a fixed grade level , the criterion 
often used, can be misleading: successive cohorts of students in that grade may differ 
sharply—in previous reading achievement and socioeconomic status, for example— 
and hence distort the true picture of gains, particularly in areas with high inter¬ 
school mobility. 

To determine which reading approach or other educational factors are reliably 
related to reading gains, we considered a large number of hypothetical factors and 
assessed each one’s likely influence on reading achievement. For us to do so, the 
schools in our sample had to vary in reading outcomes. Fortunately, they did. 
Otherwise—if all the schools had scored about the same—there would be no way of 
judging the relative success of the different approaches to reading adopted by in¬ 
dividual schools and teachers. 


CONDUCT OF THE STUDY 

Our first task was to select the 20 elementary schools that would constitute our 
sample. This selection was performed in conjunction with LAUSD staff and with the 
assistance of community groups. The schools chosen had to demonstrate: substantial 
or consistent test score gains across 6th gr ade classes between Spring 1972 and Fall 
1975; 1 enrollment of at least 400 students, with minority children making up at least 
80 percent of the student body by Fall 1975; a balanced distribution among schools 
with predominantly Black and predominantly Mexican American enrollments; and 
a ranking in the lowest 200 (out of 436 total) on Title I criteria (which are based 
essentially on the socioeconomic status of students). Appendix A describes those 
criteria in more detail, and lists the names of the 20 schools and their characteristics 
as compared with other schools in the system. 

Thus, as would be expected, our sample schools have a higher percentage of 
minority students, 2 larger enrollments, lower socioeconomic rankings, and larger 
gains in reading test scores in successive 6th grade classes than do the aggregate of 
elementary schools in LAUSD. They started much lower than all other schools on 
6th grade reading scores (measured in percentiles on national norms) in 1972 and 
had registered much larger percentage gains by 1975. These schools also had a 
starting point slightly lower than the average 80 percent minority school in the 
district; their gains by 1975 were proportionately larger than those in other minori¬ 
ty schools. 

Test score data on individual students in the 6th grade classes of 1974 and 1975 
from the 20 sample schools, and other information on the students’ ethnic and 
family background, health problems, and elementary school attendance patterns, 
were collected in the 32 junior high schools where the student cumulative records 

1 A substantial gain was defined as an increase of at least 15 percentile points on the national norms 
for the McGraw-Hill Comprehensive Test of Basic Skills (Form Q-2); a consistent gain meant that no 
single year’s increase exceeded two-thirds of the total gain. 

2 Of Black and Mexican American students, that is. The sample schools have a lower proportion of 
Asian and Native American students than does the system as a whole. 
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are now located. Thus we were able to retrieve information on reading scores for 
grades 3 through 5 for these same students, making it possible to study their reading 
progress over time, which we believed necessary for reasons discussed earlier. No 
student names have been retained in our files. Students were traced to their tradi¬ 
tional receiving junior highs and also to schools they attended through the Permits 
with Transportation program. 

We were able to collect longitudinal data (i.e., data for individual students over 
a four-year period) for 74 percent of the graduating class of 1974 and 81 percent of 
the graduating class of 1975. For none of the 20 schools did we find data on less than 
68 percent of the students. By comparing the mean raw score (CTBS-Q2) for all 
test-takers in the 20 schools with the same figure for all students for whom cumula¬ 
tive records were located, we were able to ascertain that our complete data sample 
was highly representative of the student population of test-takers in each of the 20 
schools. Data collection procedures, response rates, and the possibility of sample bias 
are discussed in detail in App. A. 

Information on school atmosphere, management and administration, teacher 
characteristics, and approaches to reading instruction were collected by means of 
personal interviews or self-administered questionnaires among the faculty and ad¬ 
ministrators of the 20 elementary schools. In three of the schools, intensive case 
studies were conducted that permitted us to characterize differences in reading 
programs and to investigate how school settings affect reading instruction. This 
information contributed to the construction of questionnaires adminstered in all the 
schools to teachers, principals, and reading coordinators, or their equivalents. The 
survey data were then transformed into quantitative factors designed to describe 
teacher characteristics, classroom approaches to reading, implementation methods, 
classroom atmosphere, emphasis on test-taking skills, training received, teacher’s 
use of resources, and parent contacts. At the school level we attempted to measure 
principals’ support and implementation activities, use of management systems, and 
the accuracy with which principals are able to identify faculty who are effective 
reading teachers. 

Chapter 2 uses the data collected to perform an analysis of the reliability and 
validity of observed reading scores, our first analytical task. Chapter 2 also presents 
the model we employed to examine the reliability of test scores and conceptualize 
valid gains, and we comment on the possible effects of "test emphasis” in some 
classrooms on the validity of test score results. 

Chapter 3 treats the influence of school, classroom, and programmatic factors 
on reading achievement, the subject of our second analytic task. We identify factors 
that our analysis indicates did in fact influence reading achievement, such as stu¬ 
dent body composition, program and implementation factors at the school and class¬ 
room level, and the activities of teachers and parents. Quantitative findings on the 
influence of each factor on reading achievement are discussed. 

Chapter 4 summarizes several of the positive factors found in our statistical 
analysis and illustrates what they look like in practice, the intent being to lend aid 
to principals and teachers seeking to improve their own schools’ reading programs. 

Chapter 5 covers the results of our interviews with parent and community group 
members, our final analytic task. It treats our assessment of the degree to which 
parents and community representatives were involved in school affairs, their roles 
in school decisionmaking, and their possible relationship to reading progress. 
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Chapter 6 presents our conclusions and recommendations. We summarize our 
findings and comment on their applicability to future programs in the 20 schools and 
to other schools in the LAUSD. Since we analyzed data from a nonrandom sample 
of schools, the generalizations from our findings are not unambiguously applicable 
to the population from which this sample was derived; it is at least possible that the 
schools in our sample share some characteristics that systematically differentiate 
them from all other LAUSD schools or even from the population of LAUSD minority 
schools. Since we, the LAUSD, and the public are interested in the generalization 
of lessons learned in this study to other schools and other times, the reader is 
cautioned that our policy recommendations contain a substantial judgmental com¬ 
ponent. Our judgments are based on our experience with other schools and other 
school systems, the knowledge of practitioners, participants, and observers we 
talked with, our understanding of current district practices, and our acquaintance 
with the process of implementing educational innovations. 



Chapter 2 

RELIABILITY AND VALIDITY OF READING GAINS 


All 20 schools show varying gains in 6th grade reading achievement from 1972 
to 1975, some of them dramatic. While our ultimate goal is to identify school, 
classroom, and reading program factors that may be responsible for these gains, 
several preliminary issues must be settled before this substantive analysis can 
proceed. First, we must establish whether the tests used by the district are reliable 
for these predominantly minority schools, as compared with the reliability stan¬ 
dards reported by the test publishers. Second, we must determine whether the gains 
are valid; that is to say, whether they reflect real gains in achievement by students 
who have experienced the reading programs in these schools. Finally, if any gains 
are found to be unreliable or invalid, we must develop a procedure for identifying 
reliable and valid gains to be used in the subsequent investigation of school and 
classroom factors. 

In addressing these issues, we will use data from both the student cumulative 
records and the teacher surveys. Data available from student cumulative records 
include results for several different reading achievement tests and subtests over the 
three years since the reading program began, some of which were administered 
concurrently. These data can be used to explore a number of reliability and validity 
problems, including the true gains for groups of students over time. In addition, the 
teacher surveys included an open-ended question concerning the teachers’ own 
opinions about the validity of gains shown by their schools. Their answers were 
useful for judging that validity. 


RELIABILITY 

All standardized test scores contain some amount of unsystematic, random error 
that is unrelated to a student’s true ability. Test reliability assesses the degree of 
random error found in a given test instrument; it is measured by reliability coeffi¬ 
cients whose values range on a scale from 0 to 1. Generally speaking, a test is 
considered reliable if its reliability coefficients equal or exceed 0.90. A test with a 
reliability much below 0.90 contains a fairly high degree of random error, and may 
be deemed unsuitable for investigating the causes of reading gains. 

To be sure, the standardized tests selected by the LAUSD and the State of 
California have demonstrable reliability coefficients exceeding 0.90, and in fact the 
reading tests most widely used in the district—the Comprehensive Test of Basic 
Skills (CTBS), forms R, Q, and S—all have reliabilities of 0.95 or above. However, 
the basis for these published reliabilities consists of national norming studies that 
use predominantly white or Anglo populations; separate coefficients for minority 
populations are seldom available. Given the frequent concern that tests for Anglo 
populations may not be reliable for minority populations, it is prudent to establish 
reliability levels for our specific sample of predominantly Black and Mexican Ameri¬ 
can schools. 
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Using data derived from student cumulative records, we can investigate two 
specific types of reliability. First, internal consistency reliability can be established 
by using the correlation between the verbal and comprehension subtests available 
from the CTBS form Q2 administered to district 6th graders in the Spring of 1975. 
Second, we have a rare opportunity to assess parallel form reliability for the CTBS 
form Q2 and form S2, both of which were administered to all district schools during 
the Spring of 1975. Both types of reliability can be calculated for each school in our 
sample and compared with similar coefficients based on norming data published by 
the test developers (see CTBS, 1970, 1974). 

Internal Consistency Reliability 

Internal consistency reliability is usually estimated by using all individual test 
items and applying a formula such as KR-20 (Guilford, 1954). By such a procedure 
the CTBS publisher reports reliabilities of 0.95 for the Q2 and R2 reading tests and 
0.96 for the S2. Unfortunately, individual item scores are not available for the 
district data, so we must adopt another method. Since the CTBS reading test is 
composed of two subtests, vocabulary and reading comprehension, both of whose 
scores are available from the cumulative record, an alternative method is to apply 
the Spearman-Brown formula to the subtest correlation (Guilford, 1954). The Spear¬ 
man-Brown formula gives the reliability for a test as p = 2r/(l + r), where r is the 
correlation between any two subtest scores whose sum constitutes the total test 
score. The subtest correlation reported by the test publisher is 0.82 for the CTBS Q2 
given to 6th graders, which yields a Spearman-Brown reliability of 0.90 for the total 
test 

Table 2.1 lists the subtest correlations for each of the 20 schools; correlation for 
the total sample is 0.80 with an associated reliability of 0.89, which is nearly identi¬ 
cal to the test norm. Further, the correlations and reliabilities for each school cluster 
fairly tightly around this value. Given a population correlation of 0.82, correlations 
computed from sample sizes around 80 (a conservative average for these schools) 
yields a 90 percent confidence interval between 0.75 and 0.87, which means that we 
would expect 90 percent of the sample values to fall in this range over repeated 
samplings. Indeed, we see that the correlations for all but two schools, or 90 percent 
of the sample, fall within this range. The two exceptions are Eastman with a correla¬ 
tion of 0.74, and Ascot with a correlation of 0.71. In general, the levels of internal 
consistency reliability are high and compare favorably with the results from test 
norms. 

Parallel Form Reliability 

Another method for estimating reliability relies on the existence of parallel 
forms, which in this case means reading tests that cover identical subject areas but 
whose individual item content differs. The correlation between two parallel forms 
is the reliability of either form. Forms R and Q of the CTBS reading test were 
designed as parallel forms. Form S is a newer version and is not strictly a parallel 
form for Q or R, since it covers some slightly different content. Nonetheless, test 
norms show a correlation of 0.93 between S and Q reading at 6th grade, which 
actually exceeds the parallel form correlation of 0.89 between R and Q. 

In the Spring of 1975 all district 6th grades were given both the Q2 and the S2 
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Table 2.1 

Internal Consistency Reliabilities for 6th Graders 
Taking the CTBS Q2 in 1975 


School 

Correlation Between 
Vocabulary and 
Comprehension Subtests 

Reliability 3 

(N) 

Alta Loma 

0.80 

0.89 

(80) 

Angeles Mesa 

0.77 

0.S7 

(75) 

Ascot 

0.71 

0.83 

(85) 

Dacotah 

0.82 

0.90 

(67) 

Eastman 

0.74 

0.85 

(HO) 

Ford 

0.75 

0.86 

(62) 

Harrison 

0.86 

0.92 

(52) 

Hillside 

0.78 

0.87 

(44) 

Manhattan 

0.76 

0.86 

(78) 

Miller 

0.8 6 

0.92 

(74) 

118th Street 

0,87 

0.93 

(62) 

112th Street 

0.87 

0.93 

(50) 

Pacoima 

0.78 

0.88 

(106) 

Rosemont 

0.81 

0.90 

(87) 

Rowan 

0.79 

0.88 

(28) 

McKinley 

0.75 

0.86 

(39) 

Sierra Park 

0.83 

0.91 

(129) 

10th Street 

0.80 

0.89 

(61) 

28th Street 

0.86 

0.92 

(86) 

Vermont 

0.83 

0 91 

(64) 

Total sampl 

e -0.80 

0.89 

(1440) 

Test norms 

0.82 

0.90b 



a From the Spearman-Brown formula p = 2r/(l+r), where r is the 
subtest correlation. 


The internal consistency coefficient given by the publisher, based 
on all individual test items, is 0.95 (using KR 20). 


reading tests, the scores for which are available in the cumulative records. Table 2.2 
shows the correlation between the Q2 and S2 reading scores for students who took 
both tests (which includes about 90 percent of our total sample). For the sample as 
a whole the Q2-S2 correlation is 0.90, which compares favorably with the norm value 
of 0.93. Again, the reliabilities for individual schools are high, although a number 
of schools fall outside the 90 percent confidence interval of 0.89 to 0.95 (assuming 
an average sample size of about 65 students). Nonetheless, all of these correlations 
but one (Manhattan’s 0.82) establish a high degree of school-level reliability for both 
the S2 and Q2 versions of the CTBS reading tests. 

Summary 

These results offer solid evidence that the reading tests used by the district in 
these minority schools meet the reliability standards determined by the test publish¬ 
er in its national norming studies. Not only are the reliabilities high, nearing or 
exceeding 0.90, but two different methods of determining reliability yield nearly 
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Table 2.2 


Parallel Form Reliability for 6th Graders 
Taking the CTBS Q2 and S2 Reading Test 
in 1975 


School 

Correlation 

Between 

Q2 and S2 
Reading Scores 

(N) 

Alta Loma 

0.88 

(63) 

Angeles Mesa 

0.86 

(72) 

Ascot 

0.91 

(78) 

Dacotah 

0.89 

(64) 

Eastman 

0.88 

(110) 

Ford 

0.92 

(53) 

Harrison 

0.90 

(50) 

Hillside 

0.92 

(44) 

Manhattan 

0.82 

(72) 

Miller 

0.86 

(70) 

118th Street 

(a) 

(a) 

112th Street 

0.91 

(49) 

Pacoima 

0.93 

(97) 

Rosemont 

0.89 

(86) 

Rowan 

0.94 

(20) 

McKinley 

0.92 

(20) 

Sierra Park 

0.90 

(126) 

10th Street 

0.87 

(58) 

28th Street 

0.88 

(80) 

Vermont 

0.92 

(55) 

Total sample 

0.90 

(1272) 

Test norms 

0.93 



NOTE: The S and Q are not strictly parallel, but 
the norm correlation for S and Q resembles the 
parallel form correlation for R and Q. 


a Data for form S is not available in our file. 


identical results. Even more important, high reliabilities are also shown on a school- 
by-school basis. 

In concluding that the tests are reliable, however, we are not yet claiming that 
the test results in these 20 schools reflect true reading gains. Reliability coefficients 
assess random, unsystematic errors or inaccuracies, but not systematic biases. In 
other words, reliability coefficients reflect the consistency of results, not necessarily 
their truthfulness. The presence or degree of systematic bias is determined by a 
validity analysis. 


VALIDITY OF GAINS 

While there are many aspects of test validity, we are concerned here with only 
one; namely, whether the changes in successive 6th grade reading test scores reflect 
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real gains in reading achievement by the students in these schools. We do not 
attempt to assess the validity of the standardized tests themselves; such investiga¬ 
tion is outside the scope of our study. Rather, we assume that the CTBS reading test, 
properly administered and scored, is a valid measure of English reading skills for 
the purpose of comparing one minority student (or school) with another. 

Even with this assumption, however, there are still two sources of potential 
invalidity for reading gains. First, although it is true that all schools in our sample 
show increasing 6th grade scores from 1972 to 1975, these apparent increases derive 
from comparing different groups of students—that is, different 6th grades from one 
year to the next. Such gains owe partly to differences in the ability and composition 
of the successive 6th grade groups. For example, the 1975 6th graders at a school 
may have scored higher than the 1974 6th graders merely because they were higher- 
achieving students to start with. To assess true change, we must compare the perfor¬ 
mance of the same group of students between, say, 1974 and 1975 as they passed 
through the 5th and 6th grades. 

A second possible source of bias is test exposure. It is a fundamental assumption 
of standardized achievement testing that students are not exposed beforehand to the 
test or to specific test content. The justifiable desire to familiarize minority children 
with standardized testing procedures has led to special test preparation exercises, 
including use of "practice” tests similar in format—but not content—to the CTBS. 
Such exercises are both legitimate and often necessary. However, any exposure of 
students to the official test or to a word list taken from the specific CTBS form would 
invalidate all or part of the test results. Any resulting gains would be real in the 
sense that scores would change, and they might also be reliable (if exposure occurred 
for both parts of a test or for both parallel forms). But such gains would be invalid 
since in all likelihood they would consist mostly of temporary changes induced by 
knowledge of the correct answers rather than an increase in reading skills. When 
test exposure occurs, the gains in reading scores are likely to be both sudden and 
very large. Since we observed such gains at certain schools, we undertook a separate 
analysis to assess this potential source of invalidity on a classroom-by-classroom 
basis. 

Change in Reading Achievement 

The first issue our validity analysis must settle is whether gains in 6th grade 
reading scores reflect improved reading ability. To do so we must compare test scores 
for a given cohort of students as it progresses through the elementary grades—for 
example, their scores from grades 3 through 6 for the 6th grade class of 1975, 
retrieved from the cumulative record. Students from the 6th grade class of 1975 who 
remained in a Title I school from 1972 to 1975 had a 6th grade Q2 reading score in 
the Spring of 1975, a 5th grade R2 score in 1974, a 4th grade R2 score in 1973, and 
a 3d grade score from the Cooperative Primary test in 1972. Since the R and Q are 
parallel forms normed in the same year, we can use them to assess changes in the 
same cohort or group of students from grade 4 to 6. It is more difficult to assess 
change from grade 3 because the Cooperative Primary is not parallel to the CTBS 
series, so test differences may not be valid indications of change. 

Before turning to an empirical analysis of reading gains, we must raise a practi¬ 
cal problem regarding the meaning of change on a standardized reading test such 
as the CTBS. Basically, there are two ways to assess change. First, gains in reading 
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knowledge can be assessed by comparing the raw scores from one year to another 
on the same form (or equivalent forms) of a given test. Obviously, such raw gain 
scores have meaning only for a particlar test, since tests vary in numbers of ques¬ 
tions, time instructions, content, and so forth. But from a policy standpoint, a more 
important limitation is that raw gains do not tell us whether a program affects 
reading ability over and above what would be expected from typical students in any 
school. The reason is that nearly all schoolchildren show raw gains in reading from 
year to year, regardless of their school or program. Consequently, we must use some 
type of standardized score to determine whether a special program has affected 
minority reading skills beyond what would be expected of average children without 
the program. 

The second way to assess change is to make comparisons between standardized 
scores such as percentiles. A percentile score—the measure used most widely by the 
district and the state—relates a student’s standing to a national norm. For example, 
a student scoring at the 50th percentile has a raw score in the middle of the distribu¬ 
tion, with half the students in the national norming sample scoring higher and half 
scoring lower. To remain at the 50th percentile in succeeding grades, a student has 
to gain about 10 raw score points a year, since the national norming sample for the 
CTBS shows this amount of gain in successive grade levels. One difficulty in inter¬ 
preting percentile scores, then, is that equal percentile scores mask absolute gains 
in reading skills. A further problem with percentile scores for this study is that the 
national norm is determined by a predominantly Anglo population, and it is general¬ 
ly known that, nationally, minority populations score lower on standardized tests 
than Anglo populations. It might be fairly argued that the minority students in 
these schools should be compared with a national norm for minorities; unfortunate¬ 
ly, separate norms for Anglo and minority populations are seldom available from 
test publishers. 

To counteract these various difficulties we present our change analysis in both 
raw score and percentile score forms. 

Total Sample Change. Table 2.3 shows the change in reading achievement 
from grade 3 to 6 for our sample of 1975 6th graders. The change for the total cohort 
appears in the top half of the table, using all students with at least one reported score 
in a given grade level, while the bottom half shows the change for a longitudinal 
sample of those students who have reported test scores in all four grades. 

The essential point to be seen from Table 2.3 is that while all the schools in our 
sample show successive gains in average 6th grade percentile scores from 1972 to 
1975, the same cohort of students shows a slightly declining trend in percentile 
scores from grade 4 to 6. In other words, for the sample as a whole, successive 6th 
grade percentile changes should not be used to conclude that a given group of 
students actually improved in reading achievement percentiles over this period. 

Table 2.3 also illustrates the problem of comparing the Cooperative Primary 
with the CTBS. Note that an increase from the 23d percentile in the 3d grade to the 
31st percentile in the 4th grade is contradicted by the slight declining trend from 
the 4th to the 6th grade. Since it makes little sense that the reading program should 
have a positive effect in the first year but not in later years, it is more likely that 
Cooperative Primary scores are not directly comparable to the CTBS series. 

We emphasize that the declining percentiles in Table 2.3 do not mean that 
students are not learning to read better. On the contrary, changes in the raw score 
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Table 2.3 

Cohort and Longitudinal Changes in Reading 
Scores for the Total Sample 


Cohort 

1972: 

Grade 3 

Coop Primary 

1973: 

Grade 4 

CTBS R2 a 

1974: 
Grade 5 
CTBS R2 

1975: 
Grade 6 
CTBS Q2 

Cohort of all 

1975 6th graders 

Percentile 

23% 

31% 

28% 

26% 

Raw meant 1 

29,5 

29,7 

38.6 

54.3 

(N) 

(1364) 

(12S7) 

(1207) 

(1502) 

Longitudinal panel 

Percentile 

25% 

33% 

30% 

29% 

Raw mean 
(N) 

29.7 

30.3 

(759) 

39.9 

55.7 


a Raw mean excludes Manhattan, Rosemont, and Sierra Park—which gave the 
Q2 in 1973—but the percentile includes all the schools. 


The maximum reading score is 85, 


means from 4th to 6th grade—9 points from grade 4 to 5, and 16 points from grade 
5 to 6—show clearly that students are advancing in absolute terms. But since 
students in the national norming sample show slightly higher gains over these grade 
levels on the same tests, the result is a slightly declining percentile trend for our 
sample. The national norms show a gain of 12 raw points from grade 4 to 5 (for 
students at the 33d percentile in 4th grade) and a gain of 18 points from 5th grade 
R2 to 6th grade Q2 (for students at the 30th percentile on the 5th grade R2). 

Not all students in the 1975 6th grade cohort took all the tests since 3d grade; 
that is the reason the number of cases differs from year to year in the third row of 
Table 2.3. Since we want to compare the same group of students over time, the 
bottom two rows of Table 2.3 show test results from a "longitudinal panel” of those 
students who have taken all the tests. Even though this panel is much smaller than 
the 1975 cohort (759 versus 1502), there is no significant bias. The panel has slightly 
higher ability than the entire cohort—about 2 percentile points—but the trend of 
decreasing percentile scores remains nearly identical. 

School-by-School Changes. Table 2.4 shows the school-by-school changes in 
reading achievement from grade 4 to 6. We do not identify individual schools here 
because we pledged confidentiality to survey and interview respondents, and some 
of the information in this section is sensitive. We have scrambled the order of the 
schools (so their names are not in alphabetical order) and have arbitrarily assigned 
letters in place of names. 

Although the 20 schools were chosen because of percentile gains in successive 
6th grade reading scores, the change analysis reveals that, for most schools, the 1975 
6th grade class actually shows a decline in average percentile scores relative to its 
position in the 4th grade. In many cases the decline is slight; fairly large declines 
have occurred for schools F, G, H, and K. Six schools show positive gains, but some 
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Table 2.4 

Change in Reading Achievement by School 


School 

Percentile Scores 

Net 

Change 

Raw Score Means 


Grade 4 

Grade 5 

Grade 6 

Grade 4 a 

Grade 5 

Grade 6 

(N) 

A 

35 

30 

30 

- 5 

39 

41 

57 

(92) 

B 

22 

18 

23 

+ 1 

24 

30 

52 

(40) 

C 

30 

32 

25 

-6 

29 

42 

53 

(12) 

D 

26 

25 

20 

-6 

27 

35 

49 

(55) 

E 

29 

22 

59 

+30 

28 

34 

70 

(42) 

F 

50 

41 

32 

-is 

42 

48 

59 

(29) 

G 

35 

29 

23 

-12 

32 

40 

52 

(60) 

H 

42 

34 

30 

-12 

37 

44 

57 

(50) 

I 

30 

25 

22 

-8 

29 

37 

51 

(48) 

J 

34 

33 

32 

-2 

32 

43 

58 

(46) 

K 

28 

20 

15 

-13 

28 

32 

43 

(33) 

L 

25 

18 

31 

+6 

26 

30 

58 

(54) 

M 

30 

26 

26 

-4 

29 

37 

54 

(22) 

N 

29 

30 

28 

-1 

28 

41 

56 

(30) 

0 

31 

(b) 

25 

-6 

37 

(b) 

52 

(55) 

P 

43 

38 

36 

- 7 

37 

47 

■ 61 

(70) 

Q 

29 

40 

32 

+ 3 

28 

48 

58 

(33) 

R 

39 

40 

52 

+ 13 

42 

48 

67 

(60) 

S 

32 

32 

30 

-2 

30 

42 

57 

(39) 

T 

26 

31 

30 

+4 

27 

42 

57 

(42) 

Total 

33 

30 

29 

-4 

30 

40 

56 

(857) 


a SchooIs A, O, and R have form Q2 scores in grade 4; all others ar? R2. 
1;> No 5th grade scores available. 


gains are slight and the pattern is not consistent over the three years. Large gains 
are shown by school Q from grade 4 to 5, while schools E, L, and R show large gains 
from grade 5 to 6. Clearly, comparing different groups of students in the same grade 
from one year to another does not constitute an assessment of the actual change or 
improvement experienced by the same group of students over time. 

Another factor that may explain part of the difference between Table 2.4 scores 
and successive 6th grade scores is that the district used median percentile scores, 
while we assigned a percentile ranking to the raw mean for a given school. The 
difficulty with median percentile scores is they do not reflect changes in scores below 
or above the median value. Our analysis of the distribution of raw scores on a 
school-by-school basis showed that, indeed, there were many shifts of raw scores 
(either upward or downward) from one year to another that occurred entirely below 
(and sometimes entirely above) the median for both years. As a consequence the 
median might remain constant from year to year, or even increase slightly, in spite 
of serious declines in scores below the median. It is our conclusion that a comparison 
of raw score means, when converted to percentiles, more accurately summarizes the 
total change over time than do median percentile scores. 

Table 2.4 also shows the changes in raw score means for each school. We empha¬ 
size that every school shows an increase in absolute reading scores in both years, 
meaning that reading skills are improving in all schools in our sample. However, 
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since the average student (at these levels) who takes the CTBS shows a gain of 12 
raw points from grade 4 to 5 and 18 raw points from grade 5 to 6, most schools show 
net declines in their percentile scores. 

While the change analysis shows declines in achievement percentiles for most 
schools, it is still possible that successive 6th grade changes are good relative indica¬ 
tors of the most effective or least effective schools. This can be tested by comparing 
the ranking of schools according to successive 6th grade percentile changes (see App. 
A) with a ranking based on Table 2.4. The rank order correlation is 0.65 for all 20 
schools, which confirms a fairly good—although by no means perfect—relationship 
between successive 6th grade change and actual longitudinal change. But when we 
exclude three schools whose scores might be invalid for reasons given in the next 
section, the rank order correlation drops to 0.51. Clearly, although 6th grade differ¬ 
ences from year to year are somewhat indicative of actual improvement, this validi¬ 
ty correlation is not high enough to warrant substitution of successive 6th grade 
change for measures of actual change. In particular, for our analysis of the effects 
of reading program factors, we will want to use actual longitudinal changes in 
reading achievement as our main criterion. 

Test Exposure 

We found that a number of classrooms showed unusually large gains in average 
reading levels from grade 5 to 6. In an analysis of individual schools and classrooms, 
we found that some students—and sometimes an entire classroom—gained over 50 
percentile points. Such changes are possible for individual students over an ex¬ 
tended period, but it is exceedingly rare for special reading programs to cause such 
large shifts for large numbers of students in a one-year period. Therefore, when the 
distribution of these unusual gains tends to cluster in certain schools, or, more 
important, in certain classrooms, the possibility of prior test exposure—"teaching 
the test”—must be explored. 

Most teachers familiar with standardized testing procedures recognize that test 
exposure invalidates test scores. Sometimes, however, the incentives to engage in 
overzealous test preparation may be so strong as to lead to test exposure. A number 
of teachers surveyed said they felt administration or principal pressure to raise test 
scores at their school. Moreover, many teachers are sensitive to the problems of 
measuring the ability level of minority children on standardized tests. When asked 
whether test scores accurately reflected their students’ ability level, 20 out 81 of our 
sample teachers reported that scores presented a worse picture than really existed 
because of problems in test-taking. As a result of these concerns, most schools have 
initiated special tests or programs for improving children’s effectiveness in taking 
standardized tests, such as creating a test awareness center in the classroom, in¬ 
structing students in test-taking skills, providing practice with multiple choice tests, 
and orienting one’s own tests to expose children to the mechanics of standardized 
tests. All these activities are entirely legitimate, appropriate, and often necessary. 

But an emphasis on test-preparation that extends to drilling children on actual 
test questions, teaching the vocabulary words on the test, or exposing children to test 
materials prior to test day introduces a serious bias and invalidates test scores. Of 
course, not all test exposure is intentional; it may occur inadvertently if a teacher’s 
vocabulary list for drill chances to have a high concentration of words from the 
various CTBS reading test forms. Unfortunately, we have evidence of intentional 
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test exposure in a few of our classrooms. Although it applies to only a small minority 
of the teachers in the study, their classrooms must be eliminated from the sample 
before we can proceed with an analysis of factors associated with valid reading gains. 

Our evidence on test exposure takes two forms. First, and most critical, three 
respondents at two schools stated that reading gains at their scools were invalid 
because of test exposure. These two schools—and especially the six classrooms in¬ 
volved—showed unusually high proportions of students gaining more than 30 per¬ 
centile points or so in the one year between the 5th and 6th grades. 

Second, several other schools with high average gains showed a concentration 
and patterning of changes similar to those where exposure was claimed directly. 
These schools are characterized by an uneven distribution of change both within and 
between classrooms, frequently yielding a bimodal distribution wherein one group 
shows little or no change and another group shows extreme change. Although these 
changes cannot be definitely attributed to test exposure, the fact that exposure 
occurred in some high-gaining classrooms prompts us to raise questions about the 
validity of reading gains for all classrooms showing a high concentration of unusual¬ 
ly high-gaining students. 

Table 2.5 shows the uneven distribution and concentration of high-gaining stu¬ 
dents. A high-gainer is defined as a student gaining more than 3 deciles (or about 
30 percentile points) in reading between grades 5 and 6. At the school level we note 
that the 143 high-gaining students are concentrated in five schools—B, E, L, R, and 
T, with percentages of 27, 73, 54, 36, and 17, respectively. The remaining schools 
have between 0 and 6 percent of these high-gainers. Mere important, the right-hand 
portion of the table shows an uneven distribution of these students within the 
schools. In schools B And T, for example, which have lower proportions of high- 
gainers than the other three, most are concentrated within 3 of the 8 classrooms. 
Altogether, 10 out of 78 classrooms have more than 30 percent of their students 
showing gains exceeding 3 deciles during the 6th grade. 

Of those 10 classrooms, 6 were in schools where claims of test exposure were 
documented in the teacher survey. For the remaining 4 classrooms we have no 
definitive proof of test exposure, although we did gather certain other indirect 
evidence in addition to the observed concentration of unusually high-gainers. First, 
for three out of the four teachers who were present in the district during the 
previous school year, we found that none of their 6th grade students gained more 
than 30 percentile points during the 1973-74 school year; in fact, two classes showed 
a net decline in achievement percentiles (the other showed a slight increase of 3 
percentile points). Thus, their large classroom gains in 1974-75 were inconsistent 
with their classroom gains the year before. Second, in one of these classrooms we 
found that two-thirds of the high-gainers had vocabulary gains exceeding 50 percen¬ 
tile points with only a few showing comparable gains in comprehension. It is possi¬ 
ble, then, that their 6th grade vocabulary gains may be spurious, possibly because 
of test preparation practice on.a word list taken in part from CTBS reading tests. 
Finally, three of these classrooms sent students to junior highs that administered 
7th grade tests in the spring of 1976. Unfortunately, since tests are given only to 
randomly selected students in each class, only 3 of the 23 high gaining students had 
7th grade reading test scores. But all 3 of these students showed precipitous drops 
in reading percentiles from grade 6 to 7 that matched their dramatic rise from grade 
5 to 6 (20% to 79% to 7%; 29% to 72% to 21%; and 11% to 69% to 19%). The only 
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Table 2.5 


Number and Concentration of Students Gaining Over 
Three Deciles During the 6th Grade 


School 

Number of 
High-Gaining 
Students 

% of Total 
Enrollment 

Number of Classrooms Having 
Indicated Percentage of High-Gainers 

None 1-15% 15-30% Over 30% 

A 

4 

4 

1 

4 


B 

14 

27 

2 


2 

C 

0 

0 

6 



D 

1 

2 

4 



E 

36 

73 



4 

F 

2 

5 

3 

1 


G 

2 

3 

3 

2 


H 

0 

0 

4 



I 

2 

3 

3 

1 


J 

2 

4 

3 

1 


K 

0 

0 

4 



L 

37 

54 



1 

M 

0 

0 

2 



N 

2 

6 

1 

2 


Oa 

— 

— 

— 

— 


P 

4 

5 

3 

2 


Q 

1 

3 

2 

1 


R 

26 

36 

1 


1 2 

S 

1 

2 

7 

1 


T 

9 

17 

2 

1 

1 

Total 

143 

14 

51 

16 

1 10 


a No 5th grade scores. 


credible explanation for such extraordinary shifts for these three students is that 
test exposure inflated (and hence invalidated) their 6th grade scores. 

We conclude, therefore, that 6 of these 10 classrooms with high proportions of 
unusually high-gaining students almost certainly have invalid gains due to test 
exposure. In another 4 classrooms, the statistical patterns and inconsistency over 
three school years raise the distinct possibility of test exposure, although we have 
no conclusive proof that it occurred. While the proportion of classrooms where 
exposure may have occurred is small, ranging from 8 percent to at most 13, our 
analysis must nonetheless take into account these possible sources of invalid gains. 


PROCEDURES FOR HANDLING INVALID GAINS 

A meaningful assessment of characteristics of effective reading programs re¬ 
quires a reliable and valid criterion for reading gains. While we have shown that 
the reliability of the CTBS reading test is adequate, two problems hamper a correc¬ 
tion for the possibility of invalid gains in 6th grade reading scores: (1) In 4 of the 
10 classrooms in which unusually large gains occurred, we have no teacher reports 
of test exposure to confirm the statistical evidence. (2) Even teachers whose overzeal- 
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ous preparation exercises led to test exposure may also have used highly effective 
approaches to reading instruction. That is, the reading test results may be a mixture 
of exposure effects and educational effects. In fact, our analysis of the characteristics 
of those teachers whose students registered unusually large gains reveals that some 
of the teachers also scored high on various classroom variables we measured. It is 
possible, then, that correction for invalid gains tends to exclude unusually good 
teachers as well. 

Faced with this dilemma, we have followed a procedure designed to reveal the 
power of school and classroom factors as a function of one’s belief about the true role 
of test exposure in producing measured gains. The procedure is as follows. We first 
measure the effect of the various factors under the assumption that all gains were 
valid. Next, we exclude the results for the six classrooms where exposure reportedly 
occurred. Lastly, we exclude results for all 10 classrooms including the four with 
unusually large and inconsistent gains. As pointed out in Chap. 3, the power of the 
school and classroom factors in explaining measured gains declines as more groups 
of students are deleted. The reader’s acceptance of our list of successful educational 
practices must therefore depend on the reader’s belief about the extent of test 
exposure and how many of these classrooms produced invalid gains. 


BIBLIOGRAPHY 

Comprehensive Test of Basic Skills, Form S, Technical Bulletin # 1, Monterey Park, 
California; McGraw-Hill, New York, 1974. 

Comprehensive Test of Basic Skills, Technical Report (Forms R & D), Monterey 
Park, California; McGraw-Hill, New York, 1970. 

Guilford, J. P., Psychometric Methods, McGraw-Hill, New York, 1954. 



Chapter 3 

INFLUENCE OF SCHOOL AND CLASSROOM FACTORS ON 
READING ACHIEVEMENT 


School policy decisions that attempt to improve students’ reading achievement 
are based on policymakers’ beliefs about how reading can be most effectively taught. 
This chapter analyzes the experiences of our 20 sample schools to identify the most 
successful school and classroom factors in raising students’ reading levels. It is 
intended to contribute to policymakers’ information regarding effective instruc¬ 
tional policies. 

We attempt to answer two questions: First, what decision levels are most closely 
associated with increases in students’ reading scores—district level, school level, or 
classroom level? This might be called "the management question,” since if district 
management policies are to be effective, staff members who actually have "policy 
leverage” (in that their decisions can affect reading achievement) should also have 
the responsibility and the authority to use it. Our analysis of the management 
question concludes that much of a child’s reading achievement depends on the 
particular school and classroom he or she attends. 

The second question can be called the "school allocation question,” because it 
deals with how each school allocates its resources and staff energies among the many 
policies that may influence reading achievement. This question can be stated as 
follows: What combinations of curriculum, program implementation strategies, and 
classroom atmosphere will promote the greatest reading achievement? It is appar¬ 
ent that principals and teachers in the district are already considering this question. 
Our analysis of the school allocation question is intended to provide additional 
information that principals and their staff members will find useful in the difficult 
choices they have to make. We have also analyzed the question of whether principals 
have enough accurate information about instruction in each classroom to identify 
strengths and weaknesses in their schools’ instructional programs. 

We conducted parallel analyses for Black and Mexican American students, 
because of the possibility that school and classroom experiences affect these two 
groups in different ways. By undertaking parallel studies of the effect of school and 
classroom factors on the reading achievement of Black and Mexican American 
students, we take into account the likelihood that the effectiveness of school factors 
depends on the specific needs of students, and on the schools’ ability to deal with 
these needs. 

To correct for the problems with the validity of the district’s test scores raised 
by test exposure, we checked our results by comparing them for different sets of 
assumptions about the scores’ validity. A few of our results are sensitive to these 
assumptions, and the effect of differing validity assumptions will be discussed. 
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THE MANAGEMENT QUESTION: WHICH POLICIES AFFECT 
READING ACHIEVEMENT? 

The question of whether district policies, school-level decisions, approaches used 
by individual teachers, or various other factors determine the reading achievement 
of inner-city children is obviously of great importance to policymakers. Since the 
district will doubtlessly wish to allocate authority and responsibility commensurate 
with staff capacities to influence reading outcomes, we begin this chapter by examin¬ 
ing whether reading results in our 20 schools were due to policies at the district, 
school, or classroom levels, or to individual and "background” factors beyond the 
reach of school policies. 

The analysis was conducted for slightly more than one thousand students who 
were in the 1974-75 6th grade class in our 20 schools. (See App. A.) We analyzed the 
increase in reading achievement that occurred over the 6th grade year, using data 
from those students who took reading tests at the end of the 5th grade and again 
at the end of the 6th, who were enrolled in regular classrooms, and for whom 6th 
grade attendance data were available. We studied this group for two reasons. First, 
it was possible to collect data for them (primarily test scores, student background 
measures, and classroom attributes). Second, the 1974-75 school year was the second 
operational year of the School Preferred Reading Program and therefore offered a 
fairer test of its influence on reading than did the previous year. 

As Chap. 2 amply demonstrates, students’ reading achievement in our sample 
varies widely. Overall reading ability increased at all 20 schools—substantially at 
some schools, very little at others (see Table 2.4). It can be inferred that school 
district policies produced an environment in which substantial increases in reading 
achievement were possible, but th-.-se policies produced results that were far from 
uniform—not a surprising outcome for a large school district. 1 This wide diversity 
in reading achievement suggests that one needs to examine differences among 
schools and classrooms to answer the management question. 

Our study considered three additional parts of the management question: 

1. How important to a student’s achievement is the particular school he or 
she attended, compared with other schools? 

2. How important is the particular classroom to which he or she was assigned, 
compared with other classrooms? 

3. How important is the student’s particular classroom within a given school, 
compared with other classrooms in that school? 2 

Each of these questions considers one particular way in which the child’s school 
might affect his or her achievement. It is apparent that any and all school effects 

1 It was technically impossible for us to determine the influence of district-wide policies on reading 
achievement. This is because we did not compare Los Angeles with other school districts on that basis. 
The amount of diversity we found, however, clearly confutes the hypothesis that district policies uniform¬ 
ly dominated reading outcomes. District policies may be a necessary condition for reading gains, but they 
are not sufficient to explain the diverse gains that were recorded. 

2 These questions were examined by testing a "baseline” hypothesis that a child’s school and classroom 
do not explain differences in learning, and comparing it with the hypothesis that they have unique and 
identifiable effects on learning. (Table 3.1 presents results of these comparisons for Black and Mexican 
American children.) This analytical technique was first used to analyze students’ school achievement in 
the research of Eric Hanushek (1970) and Richard J. Murnane (1975). We have relied on their work in 
the design and presentation of this section of our analysis. 
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are contingent on the capabilities and problems that each child brings to school: 
prior knowledge and skills, good or poor health, strong or weak motivation and 
interest, and the wide variety of family, peer, and community experiences that 
shape a child’s life and orientation to school. The question we ask is: Given the 
limited time and techniques that the schools can use to help children learn, to what 
extent is children’s reading achievement attributable to school and classroom fac¬ 
tors, rather than to the important nonschool factors that have shaped students’ 
educational performance before they even enter the classroom? 

We know that policies affecting reading instruction vary among schools, and 
that within schools, teachers have different approaches to reading instruction. If 
particular school and classroom policies and approaches have varying effectiveness, 
the 6th grade reading scores of students with identical initial achievement levels 
and background characteristics, but with different school and classroom experiences, 
will differ in proportion to the influence of those experiences. The multivariate 
analysis we conducted permitted us to measure the strength of school and classroom 
factors while holding constant the effects of a student’s sex, attendance at school, 
family background, ethnicity, and 5th grade reading achievement. (See Table 3.1. 
The table displays statistical tests of the size of the effects of school and classroom 
factors on reading, holding constant the effects of a student’s background and initial 
achievement.) 

As mentioned, we conducted parallel analyses for Black and Mexican American 
children. (All of the analyses reported in this chapter were likewise disaggregated 
for these two ethnic groups.) We did so because the structure of the analytic model 
may differ for children of different ethnicities. For example, we found that the two 
groups differed in the amount of variation in 6th grade reading achievement that 
was attributable to factors other than 6th grade school and classroom inputs. Such 
individual and background variables accounted for 57 percent of the variation for 
Black children and 64 percent for Mexican American children (see Table 3.1). 

Such differences necessarily imply that school and classroom factors variously 
affect students with different backgrounds. In this situation (which may be due to 
uneven measurement accuracy for different groups, as well as actual learning differ¬ 
ences) separate analysis was the most prudent course. 3 

We found that both school and classroom factors significantly affected the read¬ 
ing achievement of both Black and Mexican American 6th graders. This finding was 
not sensitive to whether some, all, or none of the classrooms where test exposure was 


3 Discussion of the effect of ethnic differences on model specification for education production function 
studies such as this one may be found in Hanushek (1970) and Boardman (1975). 

The educational literature on the learning patterns of Mexican American students discusses several 
issues relevant to our findings here. First of all, the language abilities of Spanish-speaking children vary 
all along the continuum of Spanish-English bilingualism and various specific cultural and subcultural 
backgrounds. (P. A. Zirkel, "Spanish-Speaking Students and Standardized Tests,” Urban Review, Vol. 5, 
No. 6, June 1972, pp. 32-40.) It therefore may be necessary to disaggregate a Mexican American sample 
such as ours into smaller subgroups based on English language ability, language spoken in the home, etc., 
before it is possible to determine precisely what factors contribute to improved reading achievement for 
these students. 

A second point is that our dependent variable is based on tests that measure reading achievement 
in English, not reading achievement per se. A number of researchers have administered standardized 
reading achievement tests in both the original English forms and a Spanish translation to Spanish¬ 
speaking students and have found that the students scored higher on the Spanish version. This difference 
was strongest on subtests dealing with word meaning. (See ibid, for a summary of these studies.) Other 
researchers contend that standard-English to standard-Spanish translations are not enough to produce 
equivalent achievement tests. One must also be aware of cultural boundaries that restrict meaning 
within a language. 
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Table 3.1 

Hypothesis Tests for School and Classroom Effects 



Black Children 

Mexican American 
Children 

(H^) The school a child attends is sig¬ 
nificantly related to achievement. 
(Comparison of Eqs. 1 and 2) 

F( 9 ,427) " 10 * 66 

p <0.00001 

F (U.534) = 3 ' 24 
p <0.0003 

(H 2 ) The classroom to which a child is 
assigned is significantly related to 
achievement. 

(Comparison of Eqs. 1 and 3) 

F {27,410) = 7,75 

p <0.00001 

F (37,508) = 2,64 
p <0.00003 

(Hg) The particular classroom within a 
particular school to which a child 
is assigned is significantly related 
to achievement. 

(Comparison of Eqs. 2 and 3) 

F ( 18,419) = 5,43 

p <0.0001 

F (26,519) = 2,01 

p <0.001 

Probability values (p values) show the probability of incorrectly rejecting the null hypothesis that 

H-l, H 2 , or Hg is false, respectively, for the samples of Black and Mexican American children. 

R 2 for Black R 2 for Mexican 

Children American Children 

Equation 1. Regression of 6th grade score 
on 5th grade score, sex, attendance, health 
problems, family status, father’s occupation, 
mother’s occupation, age, and whether addi¬ 
tional services were received under remedial 
reading or gifted program. 

0.57 

0.64 

Equation 2. Same as Eq. 1, adding dichot¬ 
omous variables for school attended by 
each child. 

0.65 

0.67 

Equation 3. Same as Eq. 1, adding dichot¬ 
omous variables for classroom attended by 
each child. 

0.71 

0.71 


NOTE: Classrooms with fewer than five Black or Mexican American children for whom complete 
data were available have been excluded from the analysis of that ethnic group. 


suspected were excluded. Growth in reading achievement depended on the school 
and classroom to which the child was assigned. Stated differently, if one wants to 
predict a child’s achievement at the end of a school year, and one already knows the 
child’s achievement level at the beginning of the year and has data on the child’s 
background, it is still extremely important to know which school and classroom the 
child attended. We may infer that the reason for these significant school and class¬ 
room effects is that different "inputs”—school policies and classroom experiences— 
were received by children, depending on the schools and classrooms they were 
enrolled in. We may also infer that these different policies and experiences are very 
important in determining each child’s reading achievement, even after taking ac¬ 
count of the many strong nonschool factors that obviously affect student’s learning. 
This finding confutes the conclusion of several cross-sectional studies, including the 
Coleman Report, that school differences are only slightly related to achievement 
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once background variables have been taken into account. (See Mosteller and Moyni- 
han, 1972, pp. 15-16, 19-22.) 

The answer to the management question, then, seems to be that both individual 
schools and individual classrooms are affecting students’ reading achievement lev¬ 
els. Certain schools and classrooms in our sample produce students whose reading 
achievement is higher than that of students elsewhere who started with similar 5th 
grade scores and backgrounds. This finding is consistent with the district’s policy of 
goal-setting for reading at the individual school level, and with granting substantial 
responsibility, authority, and some resources to schools as they work to carry out 
their programs. Because schools in the district are clearly diverse, it makes sense 
to permit them substantial autonomy in managing their affairs; this policy is consis¬ 
tent with our findings on the ability of schools and teachers to pursue policies that 
produce reading gains for their students. 4 

Such diversity points up the impossibility of formulating a "quick fix” policy to 
promote equal reading gains in all of the district’s classrooms. (Our analysis of the 
factors that produced improved reading achievement also suggests that no massive 
programs or single strategies can raise reading scores all at once.) However, it may 
be possible to improve reading on a school-by-school and classroom-by-classroom 
basis. 

The finding that the classroom to which a child is assigned is a significant 
predictor of his or her progress during the school year permits an analysis of in¬ 
dividual teachers’ contributions to students’ learning gains. (If there had turned out 
to be no differences among classrooms with respect to students’ gains, then it would 
not be possible to compare teachers’ classroom reading activities’to see which are 
most effective, as we shall do in the next section.) Our findings regarding the man¬ 
agement question, then, are that classroom outcomes are diverse and that classroom 
policies are important contributions to reading achievement. The next section dis¬ 
cusses the nature and magnitude of these contributions. 


THE SCHOOL ALLOCATION QUESTION: HOW SHOULD 
RESOURCES AND ENERGIES BE ALLOCATED? 

The success of the reading program critically depends upon discovering how to 
use the limited supply of school resources and staff energies in a school to the 
greatest positive effect. Since principals and school staff members must constantly 
choose among many policies for enhancing reading achievement, we examined the 
relationship of reading to a number of such policies. We focused on school "inputs”— 
the things that may affect reading growth—that are actually under the control of 
the individual school, its teachers, and its community. We will discuss four such 
categories of reading inputs: 


4 If we had found that a child’s school and classroom did not contribute to determining his or her 
reading achievement, we would have concluded that there was no basis for allocating the responsibility 
and authority for reading programs to the schools. The fact that our finding is that schools and classrooms 
do matter is consistent with allocating responsibility to the schools, but it is not sufficient evidence that 
this should be done. This is because we could not investigate the effects, if any, of varying the district’s 
policies regarding school authority and responsibility, since these policies did not vary in our sample. In 
particular, we did not consider the effects of policies that could hold individual schools accountable for 
students’ gains, if the schools were to be given additional authority over their programs. 
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• The teacher attributes that shape the instructional process, 

• The classroom setting in which reading is taught, 

• The curricular and instructional methods used, and 

• The implementation of programs for reading instruction. 

We conducted our. analysis for slightly less than 400 Black students who were 
6th graders in 1974-75. We analyzed the increase in reading achievement that 
occurred during the 6th grade, using data on students who took reading tests at the 
end of the 5th and again at the end of the 6th grade, and who were enrolled in 
classrooms whose teachers completed a lengthy questionnaire that probed ap¬ 
proaches to reading instruction, attitudes, and other attributes. The questionnaires 
allow us to examine in detail which classroom factors produced reading gains; our 
results depend directly on the cooperation of 81 teachers who took the time to 
answer our questions. (Two teachers did not complete the questionnaire. Because we 
lacked information about their classroom practices, it was necessary to exclude their 
students from our analysis.) 

We conducted equally extensive analyses for Mexican American students, but 
were unable to uncover statistically significant relationships between their achieve¬ 
ment and the school inputs we measured. Since we know that school and classroom 
differences are important determinants of their reading achievement, we conclude 
that we were unable to discover what policies account for their learning. This may 
be due to poor measurement, erroneous model specification, or insufficient informa¬ 
tion (perhaps on students’ English fluency). 

The analysis considered the effects of a large number of school and classroom 
variables, while statistically controlling for each child’s starting point (the 5th grade 
score), attendance, any significant health problems, and the child’s age, sex, and 
family background. 5 To correct for validity problems raised by test exposure, we 
excluded students in the six classrooms where our field research indicated that test 
exposure almost certainly occurred. (A discussion of the sensitivity of our results to 
this correction is presented below.) 

The school and classroom variables that we measured reflect four kinds of 
educational "inputs” to student reading achievement: teacher attributes, the class¬ 
room setting, instructional methods, and the implementation of the reading pro¬ 
gram. 

We investigated the effects of variables related to each of these factors on 6th 
grade reading achievement, after controlling for the effects of prior achievement and 
student background. Since prior achievement and student background obviously 


5 The influence of various groups of school and classroom factors on students’ reading achievement 
was estimated using a multiple regression analysis. The dependent variable in the regression equation 
was a student’s 6th grade reading score; independent variables were the student’s initial achievement 
(5th grade score), control variables (the student’s attendance, sex, age, health, family background, and 
the supplementary nonclassroom services received), and the school and classroom factors. The existence 
of substantial multicollinearity (i.e., complex patterns of correlations among independent variables) in 
the data led us to evaluate carefully which of our measured variables reflected the same underlying 
conceptual variables, to identify representative measures of these underlying phenomena, and sometimes 
to create new measures by combining responses to two questionnaire items (by multiplication or by linear 
combination). These efforts were guided by our categorization of school inputs into teacher attributes, 
classroom setting, instructional methods, and program implementation. We attempted to select meaning¬ 
ful and representative variables from categories, within each of which we anticipated considerable 
multicollinearity. When we encountered high collinearity between variables in different categories, we 
investigated the theoretical and empirical reasons for the collinearity and based our decision on those 
reasons. 
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account for most of the variation in school achievement (see App. B), we are interest¬ 
ed in the incremental contribution of school and classroom inputs to students’ 
reading gains. 

We now turn to a discussion of the effects that we found to be significant con¬ 
tributors to reading achievement, beyond the effects of prior achievement and stu¬ 
dent background. Table 3.2 presents the statistical relationships between reading 
achievement and the school inputs we identified as important. 6 (A detailed descrip¬ 
tion of the variables that were entered into the multiple regression analysis can be 
found in App. C.) 

Teacher Attributes 

We analyzed two groups of teacher attributes: background characteristics and 
predispositions. We collected data on several background characteristics: the teach¬ 
er’s race and ethnicity, college attended, undergraduate major, whether any gradu¬ 
ate training was received, amount of college instruction in reading, and teaching 
experience. We found no evidence of a relationship between any of these character¬ 
istics and students’ reading achievement. Three explanations are possible. First, 
teachers’ decisions about reading instruction, and their teaching skills, may be 
unrelated to background characteristics. Second, it is possible that a teacher’s back¬ 
ground influences reading, but in an interactive fashion; for example, a teacher’s 
undergraduate major in English might raise reading achievement only if the teach¬ 
er used literature and creative writing as part of the curriculum. Since we did not 
analyze any of the possible interactive effects of teacher background, we could not 
detect any such influences, even if they existed. Third, we know that teacher char¬ 
acteristics come in clusters; for example, many of the teachers who graduated from 
California State University at Los Angeles also majored in academic subjects, rather 
than in education. As a result of this clustering, it is not possible statistically to 
identify separate effects of the different characteristics, even if those effects exist. 
(There may well be effects of teacher background characteristics on student out¬ 
comes that we did not measure—for example, students’ self-concept. We restricted 
our analysis to the influence of school factors on reading achievement.) 

We also measured one aspect of teachers’ individual attitudes toward teaching 
in minority schools: their sense of efficacy in dealing with minority students. 

Our measure of teachers’ feelings of classroom efficacy is based on two ques¬ 
tions. 7 One asked whether the teacher felt that "when it comes right down to it, a 
teacher really can’t do much (because) most of a student’s motivation and perfor¬ 
mance depends on his or her home environment.” The other asked whether the 
teacher thought that "if I try really hard, I can get through to even the most difficult 
or unmotivated students.” Responses to these questions were combined into a single 
measure of efficacy—the extent to which the teacher believes he or she has the 
capacity to produce an effect on the learning of students. The more efficacious the 
teachers felt, the more their students advanced in reading achievement. This mea- 


6 Table 3.2 presents regression coefficients and t-statistics. Regression coefficients measure the in¬ 
crease in the dependent variable (6th grade score, measured on a 100-point test) produced by a one-unit 
increase in each independent variable (the school inputs). The t-statistics measure statistical significance, 
which we have coded with asterisks. 

T The standard discussion of efficacy, on which we based our instruments, is J. B. Rotter (1966). 
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Table 3.2 

Factors Producing Improved Reading for Black Children 


Factor Cocf. t-stat. Mean S.D. 


TEACHER ATTRIBUTES 


Sense of efficacy 

0.31 

2.54** 

17.1 

6.6 

CLASSROOM SETTING 





Disruptions 

-1.75 

- 3 21 * * * 

2.2 

1.2 

Number of parent visits 

0.05 

1,72** 

30.9 

30.9 

PROGRAM CONTENT 





Variety of materials, with amount 
of training (linear combination) 

0.65 

1.81** 

2.8 

2.1 

IMPLEMENTATION STRATEGIES 





Teacher adaptation of program 

0.11 

1.39* 

19.8 

9.6 

Teacher-to-teacher consultations 

0.66 

1.41* 

5.0 

1.6 


R 2 = .695 

o 

Unconstrained R = .581 
N = 356 


NOTE: Table based on regression analysis of 6th grade CTBS-Q2 raw reading 
score on 5th grade CTBS-R2 raw reading score, sex, attendance, health problems, 
family status, father’s occupation, mother’s occupation, age, whether additional 
services were received under remedial reading or gifted programs, and the variables 
displayed above. Unconstrained R2 is based on the equation with background 
variables only, excluding those displayed above. 

Statistical confidence levels for t-statistics: 

*p <0.10 

**p <0.05 

***p <0.01 


sure was strongly and significantly related to increases in reading. Obviously, teach¬ 
ers’ sense of efficacy is only one part of the morale and commitment to teaching that 
we presume is a major influence on learning. Our finding that efficacy affects 
achievement demonstrates the importance of these predispositional factors for effec¬ 
tive teaching. 

The data do not enable us to determine whether it is possible to raise teachers’ 
feelings of classroom efficacy by providing more training, support, or supervision for 
those who wish it. It is possible that an improvement in teachers’ morale and 
commitment could be produced by school policies that support teachers and help 
them solve their classroom problems, and that their sense of efficacy might improve 
as a result. (Another possibility is that the teachers with lower feelings of efficacy 
are factually describing a problem in the way they teach that is not subject to 
improvement without, for example, a change in the kind of school or classroom to 
which they are assigned, or some other substantial change in the nature of their 
work.) 
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Teachers can affect students’ reading gains in many other ways; the variables 
we have just discussed refer only to their predispositions and background attributes. 
Other aspects of teachers’ inputs to student achievement—such as their teaching 
skills, classroom management, and expertise in individualization—are discussed 
under other headings, though they clearly are important parts of teachers’ influence 
on children. 

Classroom Setting 

Because classrooms are highly diverse and complex social and instructional 
environments, we attempted to analyze the effect of the classroom setting on stu¬ 
dents’ reading progress. We asked teachers to tell us about several aspects of last 
year’s classroom setting: the level of disruption (fights, vandalism) in each class¬ 
room, the number of parents who visited the classroom, and the number of home 
visits made by each teacher. (We regard the latter two measures as classroom setting 
indicators because they characterize the classroom situation in which a strong 
relationship of home and school experiences is made apparent to children.) After 
taking account of a student’s initial achievement (the 5th grade score), background, 
and attendance, we found (not surprisingly) that the greater was classroom disrup¬ 
tion, the smaller was the growth in reading achievement. Disruption may be caused 
by only a few students, but it affects the whole classroom. Valuable instructional 
time is lost as teachers strive to restore order and resolve discipline problems. To 
the extent that school policies and resources can be used to help teachers prevent 
disruption and establish a climate of orderly and productive learning, the improved 
classroom setting can be expected to increase reading achievement. 

Greater numbers of parent visits to the classroom also were associated with 
higher levels of reading progress. 8 For our sample of 6th grade teachers, the median 
number of visiting parents was 15 (the median number of home visits was two). We 
speculate that the relationship between parent visits and reading achievement 
reflects teachers’ varying success in drawing parents into the educational process. 
We examined whether the individual students whose parents met with the teacher 
had improved reading scores; they did not. Parents visiting in classrooms apparently 
affect students as a group; this finding contributes to our belief that parent visits 
reflect the atmosphere of a classroom. 

If the foregoing speculations are accurate, a policy that strongly encouraged 
parent visits might not necessarily produce increased reading achievement. This is 
because the number of visits may be simply an indicator of certain teachers’ 
capabilities for enhancing student motivation and work habits by involving parents. 
If so, merely mandating the use of visitation would not automatically raise other 
teachers’ capabilities to affect these processes. However, it is clear that classrooms 
that were visited by parents received a benefit in reading achievement that applied 
to all of its students. 


8 Because parent visits and teachers’ home visits were highly correlated, r = 0.89, and appear to 
measure the same kind of classroom atmosphere, we concentrated our quantitative analysis on parent 
visits. 
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Curricula and Instructional Methods: Substance of the Reading 
Program 

A teacher’s reading program may affect achievement for two reasons: because 
of its specific content and approaches, or because of the way it is applied by an 
individual teacher. Without direct observation of each teacher’s combination of 
teaching methods, techniques for reading instruction, grouping practices, and use 
of materials, only summary measures of teachers’ instructional activities can be 
gathered. The enormous diversity of pedagogical approaches being used in our 20 
schools further complicates any statistical analysis based on a restricted number of 
teachers. While recognizing these serious limitations, we collected data on several 
aspects of each teacher’s reading curriculum and instructional methods. 

Our data enabled us to consider the effect on reading instruction of the following 
curriculum elements: 

• Techniques for reading instruction (individual diagnosis, word-attack, com¬ 
prehension, and supplementary reading methods, 9 as well as the amount 
of time spent on reading instruction), 

• Nontraditional teaching techniques (team-teaching and open classroom 
approaches 10 that deviate from traditional classroom methods), and 

• Teachers’ use of a variety of materials for different students at the same 
time. 

In general, the curriculum measures that we investigated were not systemati¬ 
cally related to better or worse levels of reading achievement. The absence of signifi¬ 
cant statistical relationships probably means that the curricula were used in some 
effective ways and some ineffective ways, and that on balance there was no predict¬ 
able effect due to a teacher’s merely having used a particular curriculum or tech¬ 
nique. An important exception to this general conclusion is our finding that teach¬ 
ers’ use of a variety of materials was strongly related to increased reading achieve¬ 
ment. This finding relates to the way teachers actually apply a teaching technique, 
rather than to particular characteristics of a technique or approach. 

Because there is a great deal of variation in the ways that any particular reading 
curriculum can be carried out in classrooms, we attempted to formulate a measure 
of the actual behavior of teachers in relation to the most frequently emphasized 
instructional approach to reading, that of individualized and diagnostic instruction. 
We asked teachers how much their school’s reading program had actually affected 
their "use of a greater variety of materials for different children at any one time.” 
Our case studies provided evidence that this behavioral measure reflected an impor¬ 
tant area of reading instruction: the extent to which teachers deal effectively with 
the complex and difficult operations required for individualization. Our finding that 

9 Individual diagnosis techniques emphasize the specific, highly disaggregated skill weaknesses that 
each child has as he or she progresses. Word-attack techniques are aimed at improving a child’s ability 
to decode (or "sound out”) unfamiliar words, usually emphasizing phonics. Supplementary reading 
techniques emphasize comprehension through spelling, study skills, and the teaching of reading in other 
subjects, such as social studies. Comprehension strategies use story reading and oral reading to increase 
students’ understanding of what they read. 

10 Team-teaching is a technique in which two or more teachers share the responsibility for their 
classes, with each teacher teaching certain subjects to all of the students. Open classroom approaches 
involve more child-centered, independent-study tasks than the "teacher-centered” traditional classroom 
in which teacher assignments and the lecture method predominate. 
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a teacher’s use of a greater variety of materials was positively related to reading 
achievement supports the interpretation that an individualized curriculum (when 
thoroughly put into practice) produces higher achievement. The likely importance 
of effectively implementing such a reading program led us to inspect the relationship 
of varied materials usage to the amount of training received by teachers; in addition, 
their correlation is substantial, perhaps because it takes new ideas and skills for 
teachers to effectively use diverse materials simultaneously. When we combined the 
effects of varied materials usage and teacher training (to reflect the interdependence 
and the frequent joint occurrence of these two classroom inputs), we found a very 
large and significant relationship with reading achievement. (See Table 3.2.) 

The numerous specific curriculum measures for which we did not identify signifi¬ 
cant relationships to reading outcomes included the following: 

1. Various techniques for reading instruction. For the four major clusters of 
reading strategies we investigated (individual diagnosis, word-attack, comprehen¬ 
sion, and supplementary reading), our general finding is that the use of any particu¬ 
lar approach seems not to be associated with higher or lower progress in reading. 
(Our measure of individual diagnostic techniques was much less strongly related to 
reading achievement than was the use of varied materials, despite their conceptual 
similarity. We hypothesize that variety of materials is a more "behavioral” measure 
of actual individualization, as compared with teachers’ reports of whether so-called 
individualized techniques are in use.) 

2. Nontraditional teaching techniques. We did not discover a significant rela¬ 
tionship between either team-teaching or open classroom approaches and students’ 
reading achievement. In other words, some students benefited from these teaching 
techniques and others did not; on balance, no systematic effect on reading achieve¬ 
ment was produced. 

It would be incorrect to conclude that these techniques are not useful. With the 
exception of the extent to which varied materials are used for different students, 
which proved to be a strong and effective strategy, we would conclude that there is 
a great deal of variation in the effectiveness with which various reading programs 
have been used in our sample classrooms. 

We also found no significant relationship between achievement and time spent 
on reading instruction. (Reading time per day ranged from 45 minutes to two hours.) 
We would suggest an interpretation of this finding parallel to our explanation of the 
effect of reading strategies: we do not know that more instruction does not help, but 
we do believe that there is great variation in how effectively instructional time is 
spent. The implication is that administrators and teachers should concentrate on 
policies for enhancing the quality of reading instruction, rather than mandate an 
increase in its quantity. It appears that the quality of a reading program is not 
necessarily determined by its content or its quantity (within limits). Instead, quality 
seems to depend on how well the school’s reading program—whatever it is—is 
adapted to the needs and capabilities of each teacher and classroom. This link 
between a reading program and student achievement can be good or bad, effective 
or ineffective, for a wide variety of specific reading programs. 

Implementation of Reading Programs 

We investigated the ways in which the 20 schools implemented their reading 
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programs. By implementation we mean the policies and activities that affected how 
easily, how flexibly, and how thoroughly the reading program was carried out by 
teachers in each school (as opposed to the specific instructional content of the read¬ 
ing program). We studied implementation because educational programs are not 
self-propelling; they do not automatically begin to function in every situation just 
as their designer intended they should. Instead, each teacher must find practical 
ways to carry out the reading program, taking into account his or her strengths and 
weaknesses, students’ capabilities, interests and motivations, and the support and 
facilities made available by the school. The various ways in which the new reading 
programs were installed—with or without the use of training, for instance—varied 
greatly for the teachers and schools in our sample. We found that implementation 
decisions do affect the level of achievement produced by reading programs. 

The role of teachers in implementing a reading program can be either large or 
small, for any particular kind of reading program. In examining the influence of the 
teacher’s role in implementation on improvement in reading, we collected informa¬ 
tion on whether teachers were encouraged to conform to a predetermined program 
or asked to contribute their own ideas. Students’ reading achievement was reduced 
where teachers felt that their reading instruction was expected to conform closely 
to the school’s reading program guidelines. 

Conversely, the more that teachers were encouraged to adapt or modify the 
reading program on an individual classroom basis, the more their students increased 
in reading achievement. These effects of teachers’ flexibility in program implemen¬ 
tation were quite strong. Because these two measures appear to capture a single 
dimension of teacher flexibility in implementation, and since they were highly 
correlated, we combined them by multiplication into a single measure, which is 
strongly and significantly related to improved reading: the more flexibility, the 
greater the reading achievement. See Table 3.2. (Of course, there is a limit to this 
finding: when teachers are completely independent in deciding how to teach reading, 
then there is no school-wide program, and school policies are irrelevant to classroom 
outcomes.) 

Because the teaching of reading is a difficult and complex task, many of the 
schools used some of their discretionary budget to provide training and resource staff 
to assist teachers in putting the reading program into action. We analyzed the effects 
of several of these "implementation strategies,” which attempt to increase the 
effectiveness of the reading program by helping teachers to get the most out of 
whatever reading system has been chosen. We found that additional training con¬ 
tributed to effective implementation of reading programs; as we discussed earlier, 
we combined our measure of training with the measure of teachers’ use of varied 
materials, in order to sharpen the effect of that aspect of curriculum on reading 
outcomes. We also found that teachers’ informal consultations with other teachers 
about the reading program produced significant advances in their students’ reading; 
in a sense, teachers are an expert resource for their colleagues, and the value of their 
experiences for other teachers trying to implement the reading program can be quite 
high. The relationship of teacher-to-teacher consultations to increased reading 
achievement was strong and statistically significant. The implication of this finding 
is that implementation problems can be solved by staff members in each school. 

While our research did not investigate the costs of any of these strategies for 
implementing new reading programs (and so cannot judge their cost-effectiveness), 
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we were able to document the substantial positive influence on reading that is due 
to training and teacher-to-teacher consultation. Since these approaches can be used 
with any reading program, they may present opportunities for school officials to 
assist teachers in implementing a wide range of instructional methods for reading. 
Our findings regarding the implementation of reading programs reinforce the idea 
that how a program is carried out may well be more important than its particular 
techniques and methods. 

While the school inputs we have found to be related to reading achievement are 
significant predictors of reading improvement, their interpretation depends to some 
extent on our assumptions about the validity of the district’s test scores that we 
analyzed. To show the extent to which the findings are sensitive to our corrections 
for possible invalidity of test scores, Table 3.3 presents separate analyses for three 
groups of Black students. First, we performed the analysis of effective reading 
factors for all of the Black students in our sample. Second, we obtained results for 
the sample excluding students in the six classrooms where our field research indi¬ 
cated test exposure almost certainly occurred. (This is the sample corrected for 
probable test exposure that we used as the basis for the analysis discussed above.) 
Third, we further excluded all students in the four classrooms where a large propor¬ 
tion of students made very large gains. (We had done additional fieldwork in an 
attempt to locate evidence of test exposure in each of the high-gain classrooms, but 
obtained no additional independent reports of test exposure.) As to be expected, our 
results are sensitive to which groups of students are included and which are ex¬ 
cluded; we conducted three analyses to enable us to evaluate the implications of 


Table 3.3 

Comparison of Regression Results, with Correction 
for Possible Test Exposure 


All “Exposed” “Exposed” and 

Black Students High-Gain 

Students Excluded Groups Excluded 


Factor 

Coef 

t-Stat 

Coef 

t-Stat 

Coef 

t-Stat 

Disruptions 

-2,02 

-3.6 

-1.75 

-3.2 

-0.85 

-1.6 

Number of parent visits 

0.01 

0.3 

0.05 

1.7 

0.004 

0.1 

Sense of efficacy 

Variety of materials, with 

0.34 

2.8 

0.31 

2.5 

0.37 

3.3 

amount of training 

0.95 

2.6 

0.65 

1.8 

0.66 

1.9 

Teacher adaptation of program 

0.15 

1.9 

0.11 

1.4 

-0.001 

-0.01 

Teacher-to-teacher consultations 

R2 

9 

Unconstrained R 

N 

0.89 

1.9 

0.67 

0.55 

373 

0.66 1.4 

0.70 

0.58 

356 

0.19 

0.4 

0.76 

0.73 

266 


NOTE: Table is based on regression analyses of 6th grade CTBS-Q2 raw reading score 
on 5th grade CTBS-R2 raw reading score, sex, attendance, health problems, family status, 
fathei s occupation, mother s occupation, age, whether additional services were received 
under remedial reading or gifted programs, and the variables displayed above. Uncon¬ 
strained R2 is based on the equation with background variables only, excluding those 
displayed above. 
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choosing among them. It is our judgment that the effect of school factors is most 
accurately captured by using the analysis based on the second group of students 
(excluding the six classrooms with direct field evidence of test exposure.) 11 We 
regard the sample that excludes classrooms with reports of test exposure as judi¬ 
ciously correcting our analysis for potential bias; the preceding discussion of effec¬ 
tive school factors is therefore based on it. The other two samples are extreme 
comparison cases, representing hypotheses of "no bias” (the complete sample) and 
"worst-case correction” (the sample eliminating all high-gain classrooms). As Table 
3.3 shows, the sample excluding all high-gain classrooms exhibits weaker effects 
than does the sample excluding only test-exposed classrooms. (The complete sample 
shows effects that are mostly similar to those of the second, "middle-ground” sam¬ 
ple.) The reader who judges that all the classrooms exhibiting high gains ought to 
be excluded should exercise caution in regard to our findings. The analysis based on 
the middle-ground sample strongly supports the theory and findings of comparable 
studies (particularly the Rand study of school innovations, Federal Programs Sup¬ 
porting Educational Change. This congruence of findings is, in itself, corroboration 
of the validity of our preferred sample. 

The separate effects of the school inputs we found to be important to reading are 
small compared with their combined power. As an exercise designed to illustrate the 
hypothetical overall effect of the reading approaches we have identified as effective, 
we have performed a simulated experiment based on our study results. Table 3.4 
presents the simulation, performed for samples based on both the middle-ground 
and worst-case assumptions regarding test validity. The average student in the 
samples had a raw score of about 36 on the 5th grade CTBS-Form R2. This corre¬ 
sponds to a percentile rank of about 25. Had this student attended the average school 
and average classroom among those we studied, his or her CTBS-Q2 raw score at the 
end of the 6th grade would have been about 51, which would have meant placement 
at the 22d percentile on the national norms for the 6th grade. If, instead, this same 
student had been exposed to all of the most effective school and classroom factors 
we were able to uncover, his or her hypothetical raw score would have been further 
increased by between 7 and 13 raw score points (depending on whether all high- 
gaining classrooms are excluded, or only those in which field evidence supported the 
presumption that test exposure occurred). These alternative results would have 
placed the hypothetical student at the 32d (raw score of 58) to 43d (raw score of 64) 
percentiles. Thus an optimal approach to reading instruction has the potential of 
adding 10 to 20 percentile points to students' 6th grade reading scores. In the light 
of prior research findings on the downward trends in percentile ranks of minority 
children as they grow older, our results suggest the possibility of reversing the 
expected patterns and producing a turnaround in test performance when measured 
longitudinally. 

Relating Reading Achievement to School Decisions 

The analysis of the school allocation question has focused thus far on identifying 


11 That analysis corrects our sample for the "omitted variable” of test exposure; it uses the information 
we have gathered about the actual occurrence of test exposure, without ignoring potential biasing effects 
(as the complete sample does), and without unduly restricting the variability of test scores in the sample 
(as excluding all of the high-gaining classrooms tends to do). 
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Table 3.4 

Average and Simulated Reading Scores for Black Children, 
for Alternate Assumptions on Test Validity 



Sample 

Excluding 


Classes 

All Classes 


with Verified 

with Very 

Reading Score 

Test Exposure 

s High Gains 

5 th grade 



Average raw score 

35.8 

37.6 

Average gain 

15.8 

13.4 

6th grade 



Average raw score 
Hypothetical increase 

51.6 

51.0 

from effective factors 

12.6 

7.4 . 

Hypothetical 6th grade raw 



score with effective factors 

64.2 

58.4 


NOTE: Scores are based on samples of Black children for whom 

complete data were available. “Classes with verified test exposure” are 
those six classes for which there was field evidence of test exposure. 
“All classes with very high gains” comprise four more classes for which 
there was some statistical evidence that the district’s reading scores may 
not have been valid. Hypothetical scores were computed using regres¬ 
sion coefficients in Table 3 -T Data on “effective factors” are the high¬ 
est values on the school inputs that occurred in our sample. 


the inputs that are associated with reading achievement. Another approach to the 
question of how to improve minority students’ reading instruction is to investigate 
whether adequate and accurate information is available to administrators at in¬ 
dividual schools, so that they can act to identify and correct instructional weak¬ 
nesses, and can support the strengths of the instructional program. The complexity 
and extremely large size of the LAUSD make it unproductive for many important 
educational decisions to be made centrally. (Our findings regarding teachers’ flexibil¬ 
ity in implementing the reading program cast doubt on the wisdom of such a policy, 
even if it were possible. In fact, the district’s School Preferred Reading Program 
clearly emphasizes the need for individual schools to have substantial autonomy in 
planning their reading programs.) Therefore, we have attempted to discover 
whether principals possess sufficient information to judge the effectiveness of each 
of their teachers. 12 

We asked principals to describe the performance of their 6th grade teachers on 
several dimensions of classroom style and effectiveness in reading instruction. Their 
responses represent what they believe to be the strengths and weaknesses of each 
of their teachers. These descriptions, taken as a group, were strongly correlated with 
the reading achievement of students in each teacher’s classroom. Because it was 

12 This analytical approach was first used in Richard J. Murnane’s study, The Impact of School 
Resources on the Learning of Inner City Children, Ballinger Publishing Company, Cambridge, Massa¬ 
chusetts, 1975. 



32 


difficult to identify which dimension best reflected the actual gains made in a teach¬ 
er’s classroom (because of intercorrelations among the assessments), however, we 
combined the multiple measures into a single summary of the principal’s assess¬ 
ment of each teacher. The data indicate that the sum of the principal’s descriptions 
of each teacher was a very good predictor of students’ reading achievement. (See 
Table 3.5); note that this analysis finds that principals’ assessments, both positive 
and negative, are accurate predictors of reading outcomes— not that principals’ as¬ 
sessments produce certain outcomes.) 

Clearly, then, principals have the information and the analytic capacity to 
identify their teachers’ instructional strengths as well as the problems facing them. 
Whether or not they have the time, resources, and desire to act on that knowledge, 
we cannot say. Yet one implication of this finding is that principals may have enough 
information to make policy decisions at the school level. 

We found principals’ assessments to be good predictors of student reading out¬ 
comes for both Black and Mexican American students.(See Table 3.5.) Despite our 
inability to identify statistically significant relationships between specific classroom 
inputs and the reading achievement of Mexican American children, it is clear that 
the principals of schools serving these children are able to identify effective and 
ineffective classrooms, based on their observation of each teacher. 

Answers to the School Allocation Question 

This analysis has attempted to illuminate the central allocation problem con¬ 
fronting school staff: How can the school’s finite supply of resources and staff energy 
be most effectively used to increase reading achievement? We found school factors 
that are strongly related to reading achievement, as well as numerous factors that 
show no relationship to achievement. We regard these findings as offering sugges¬ 
tions for educational policy decisions. 

Teachers matter for reading: their sense of efficacy in teaching minority children 
contributes to reading achievement. This finding is important both for teachers— 
whose investment of energy and commitment in the educational process is too 
frequently underemphasized—and for school officials who may be able to support 
the morale and commitment of teachers. 

Classroom atmosphere affects the reading gains of all of the students in a class. 
Our measurements of the classroom setting are not subtle, but they confirm that the 
background for learning can affect teachers’ and students’ productivity. Classes with 
high levels of disruption had diminished reading achievement; classrooms with 
frequent parent visitors had increased reading achievement. These measures of the 
setting created by the school and the teacher, and probably other aspects of the 
classroom setting whose effects we did not measure, are significant determinants of 
reading improvement. If there are barriers to the improvement of classroom setting 
that are remediable by using school resources and staff energies, then principals 
may be in a position to produce a reading "payoff’ by removing the barriers. 

Instructional programs can contribute to reading success, though merely intro - 
ducing a new reading system will not necessarily affect achievement. Reading cur¬ 
ricula do not, by themselves, determine a program’s success or failure. Achievement 
varies widely among classrooms that have very similar curricula and reading strate¬ 
gies. This may imply that any of the reading programs we studied can be effective 



33 


Table 3.5 

Principals’ Assessments as Predictors of 
Reading Achievement 


Item 

Black 

Students 

Mexican 

American 

Students 

Coefficient of principals’ assessments sum 

0.22 

0.10 

t-statistic 

3.24 

1.90 

Confidence level for t-statistic 

p < 0.001 

p< 0.05 

Mean, standard deviation 

39.7, 8.8 

41.0, 7.4 

R2 of equation 

0.586 

0.665 

Number of cases 

405 

500 


NOTES: Table is based on regression analysis of 6th grade CTBS-Q2 
raw reading score on 5th grade CTBS-Q2 raw reading score, sex, atten¬ 
dance, health problems, family status, father’s occupation, mother’s 
occupation, age, whether additional services were received under reme¬ 
dial reading or gifted programs, and the sum of principals’ assessments 
of each child’s classroom teacher. 

The principals’ assessment sum includes the following components, 
each measured on a quantitative scale: 

—Maintaining consistent and reasonable control and discipline in the 
classroom; 

—Planning and executing lessons and other classroom activities crea¬ 
tively and thoroughly; 

Creating an atmosphere of excitement and eagerness to learn (“turns 
kids on”); 

—Implementing the reading program as it was >^id out, according to 
guidelines and on schedule; 

—Effectively using a management system to monitor students’ acqui¬ 
sition of reading skills; 

—Individualizing reading instruction to address the needs of each 
learner (e.g., effective use of flexible grouping); 

—Effectively combining comprehension and skills instruction to create 
a balanced reading curriculum; 

Helping students to develop skills necessary for good performance on 
standardized tests; 

Developing effective learning centers in reading and other content 
areas. 


depending on how it is used; at the same time, it appears to indicate that the 
implementation of reading instruction, rather than its substance, affects success. 
The curriculum approach that did show a relationship to reading growth was the 
use of varied materials for different children at the same time; the effect of this 
variable was enhanced when it was combined with increased amounts of teacher 
training in how to apply the reading program. 

Program implementation affects reading achievement across a range of instruc¬ 
tional methods and school settings. The way in which a reading program is carried 
out strongly affects how much students learn. This is true even after taking account 
of the impact of curriculum, classroom setting, and teacher attributes. In particular, 
the more active teachers were in deciding how to implement the reading program, 
the more reading achievement improved. Our findings that both the amount of 
training and the extent of teacher-to-teacher consultations are related to reading 
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achievement may offer opportunities for improving some programs. In general, 
implementation strategies are a crucial and often overlooked component of effective 
school programs. 

Principals know which classrooms have problems and which are particularly 
effective. Our analysis demonstrated that principals’ assessments of teachers accu¬ 
rately predicted which students improved most in reading. While it is likely that 
parents and teachers will also have information about school problems, the principal 
has a unique opportunity to observe the whole situation at his or her school, and act 
to improve it. As the full-time managers of their schools’ educational programs, 
principals have the knowledge to decide when and where changes are needed. 

These findings show a consistent pattern: the effectiveness of a reading program 
depends much more heavily on school and classroom policies and decisions than it 
does on the packaged content of the program or on teachers’ background character¬ 
istics. The latter two were not among the effective factors we identified. In whatever 
reading program is selected, much more important factors appear to be orderliness 
in the classroom, teachers’ confidence in their own efficacy, and the adaptation of 
a variety of materials to perceived classroom needs. So are teacher training, infor¬ 
mal consultations among teachers of reading, and flexibility for teachers in applying 
the reading program to their particular classrooms. The most promising avenue for 
reading improvement therefore appears to lie in making a reading program effective 
at the individual school and classroom level. No one has yet devised a prepackaged 
program or set of techniques that will automatically produce gains for all schools, 
whether minority or otherwise, merely by being put into operation. 


CONCLUSIONS 

This chapter has dealt with two issues: 

* The management question of the extent to which district-, school-, and 
classroom-level decisions have differential effects on reading achievement, 
and 

• The school allocation question of how a school’s limited resources and 
energies can be allocated to produce improved reading. 

In our analysis of the management question, we examined whether or not a 
student’s school and classroom had an impact on his or her 6th grade reading 
achievement. We found that individual schools and classrooms are highly signifi¬ 
cantly related to students’ reading achievement, for both Black and Mexican Ameri¬ 
can children. In our study of the school allocation question, we analyzed the effects 
of a wide variety of factors on achievement. Most of the specific attributes and 
programmatic factors had little effect on reading, providing evidence that classroom- 
level implementation—rather than any technical solution—is the central factor in 
6th graders’ reading progress. Moreover, we were able to identify some particular 
school and classroom factors that have substantial power to influence reading 
achievement for Black students. Despite our efforts to identify such factors for 
Mexican American students, our analysis was confounded by the effects of un¬ 
measured variables (and perhaps by variation in the level of English fluency across 
Mexican American children.) Finally, we found that principals possess accurate 
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information about the relative strengths and weaknesses of their teachers in read¬ 
ing instruction, for both Black and Mexican American students. 

Our answers to both the management question and the school allocation ques¬ 
tion tend to reinforce the intent and the policies that make up the district’s School 
Preferred Reading Program. The program established substantial local autonomy 
for schools in deciding on their instructional package for reading; our analysis of the 
management question shows that there are, indeed, important effects on reading 
that originate in individual schools and classrooms. The program emphasizes teach¬ 
er involvement in implementing the reading program; we found that such involve¬ 
ment increases achievement. The program mandates parent involvement in each 
school’s reading program; we found that parent visits to the classroom are associated 
with improved reading. The program requires that schools carefully consider the 
means they would use to implement their reading program, and in particular to 
assure that adequate training and staff development are provided; our analysis 
showed the importance to reading achievement of the implementation strategies 
selected, not just of the techniques of reading instruction. Finally, the program 
places considerable responsibility for reading outcomes in the hands of each school; 
we found that principals’ information about reading instruction in classrooms 
matches the actual record of achievement. Our findings show that the policies of the 
School Preferred Reading Program are important ones for the achievement of 
minority students. 
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Chapter 4 

ELEMENTS OF A SUCCESSFUL READING PROGRAM: 
EFFECTIVE SCHOOL AND CLASSROOM FACTORS 

IN PRACTICE 


Our statistical analysis has indicated a number of factors that can contribute to 
improved reading achievement. However, limits on available resources make it 
difficult, if not impossible, to implement school- and classroom-level programs that 
will maximize the positive effect of all these factors. 5 This chapter describes several 
successful techniques, implementation strategies, and staff characteristics explored 
in our visits to the sample schools. The purpose is to show that, while no single factor 
is sufficient to produce notable gains, a combination of selected factors may do so. 
Consequently, the schools and classrooms included in this chapter were not chosen 
because they were successful in all aspects of their reading programs; rather, certain 
elements of their programs illustrate what the positive factors found in our statisti¬ 
cal analysis look like in practice. We hope this discussion will be useful to principals 
and teachers seeking to improve their own schools' reading programs. 


SCHOOL-LEVEL POLICIES AND CHARACTERISTICS 

Among school-level factors that affect reading achievement, the leadership role 
of the principal is one of the most important. As a group, the principals in our sample 
knew which classrooms had problems and which were particularly effective. How 
an individual principal acts on this kind of information constitutes part of his or her 
management task. Principals use a variety of approaches, of course, but the most 
effective principals we observed were able to achieve a balance between a strong 
leadership role for themselves and maximum autonomy for classroom teachers. 
These principals conceived of their role as promoting three broad goals: 

• To create, through example and positive reinforcement, norms that empha¬ 
size hard work and dedication; 

• To establish incentives for greater professionalism and classroom-level 
innovation; and 

• To maintain an environment that supports teacher efforts in the classroom 
and minimizes outside factors that can disrupt the learning process. 

The principals have worked to achieve these goals in a number of ways. For 
example, some principals stress the contribution the school is making to the com¬ 
munity. They continually emphasize to teachers the ways that a hard-working staff 
can strengthen the school and, in turn, serve the community well. Principals may 


1 Because none of the classrooms or schools we observed embodied all the factors discussed in this 
chapter, we do not know empirically what the effects of all these variables would be if they were used 
simultaneously. If it were practical to implement all of them, however, there is no reason to believe they 
would exert anything other than a positive effect on reading achievement. 


36 
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try to encourage greater professionalism and classroom-level innovation in the way 
they allocate extra materials and resources to individual classrooms—giving a pro¬ 
portionately larger share to teachers who wish to implement new approaches. Other 
principals have used a faculty newsletter to discuss new and effective techniques 
that particular teachers are trying, thereby highlighting innovative approaches and 
reinforcing teacher initiative. 

Those principals who were most effective in maintaining a school environment 
supportive of teacher efforts visited individual classrooms often (perhaps once a 
week) and went there with a specific purpose in mind. For example, they might 
observe whether learning centers had been established or whether students had 
been given clear guidelines for working independently. On some visits, the principal 
was accompanied by the reading coordinator or a reading resource teacher. In any 
event, the principal would try to coordinate his or her perceptions of teacher needs 
and teacher requests for additional material or assistance with the reading special¬ 
ists. The teachers therefore knew there was administrative concern with and inter¬ 
est in what went on in the classroom. These visits also provided information to 
principals about the kinds of support that would be most useful to particular teach¬ 
ers. Our analysis indicated the importance of classroom visits by parents. Here again 
principals can take constructive action. They can encourage parental and commun¬ 
ity involvement, and with the right kind of programs can be instrumental in actual¬ 
ly bringing parents into the school. Principals at several schools we visited have 
stimulated parental involvemnt in the school by establishing extensive activity 
programs and by encouraging teachers to seek the involvement of their students’ 
parents. School choirs and other extracurricular activities have been started for the 
students, and s a full program of events has been organized to involve parents, includ¬ 
ing mother-daughter teas and parental participation in school-sponsored athletic 
events, such as bowling nights and baseball games. All such activities are designed 
to bring parents to the school in an informal and unthreatening way. Principals and 
teachers at schools that have used such techniques have found that parental partici¬ 
pation in nonacademic events has often expanded into an interest in the school’s 
academic program. (The effectiveness of these approaches is further explored in 
Chap. 5, from the perspective of parents.) 

Other payoffs from these activity programs can also affect reading achievement. 
A well-rounded program of extracurricular activities often motivates more children 
to come to school regularly (and thus to reading classes). Principals believe that 
these programs raise teacher morale. Teachers see that parents are interested in the 
school, sometimes to the point of paying attention to the maintenance and cleanli¬ 
ness of the school physical plant. Activity programs to involve parents may appear 
to be related only indirectly to reading achievement, but they affect such factors as 
parental classroom visits and teacher morale, which affect reading achievement 
more directly. 


CLASSROOM ATMOSPHERE AND TEACHER ATTRIBUTES 

Teacher attributes and general classroom atmosphere are important to reading 
achievement. As indicated earlier, however, teacher attitudes are more significant 
than their background characteristics. The most effective reading teachers had a 
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strong sense of personal efficacy in teaching minority children; they believed they 
could "get through” even to children with shaky motivation or home background. 

Their confidence does not mean they ignored the ways their students might be 
disadvantaged. On the contrary, they were well aware of socioeconomic problems 
and sometimes used them to motivate their students. For example, at the beginning 
of the year, one teacher drew bar graphs on the blackboard and indicated where the 
class placed in reading based on national norms. He then delivered a pep talk to the 
class. "This is where they say you are (near the bottom of the graph) because you’re 
poor and Black. Can you think of any reason why you should be down here? I can’t, 
and I think you should be up here (close to the mean).” This exercise had a dual 
purpose: to challenge the students to perform better and to show them exactly what 
the class goals were in reading achievement. Throughout the year, the teacher and 
students referred to the graph and assessed their progress. Other teachers were less 
overt in their efforts to overcome low student motivation and home environment, 
but were still aware of these as important factors affecting student performance. 
Some teachers, for example, devised a set of classroom games to improve the self- 
concept of their students. But whatever technique these teachers chose to deal with 
the special needs of their students, they remained confident that their teaching 
would yield positive results. 

Another factor found to contribute to improved reading achievement was teach¬ 
er visits with parents (either in the classroom or at the students’ homes). Most 
teachers who worked to maximize the number of these visits believed they served 
two purposes. First of all, by meeting with parents, teachers were able to get a better 
idea of what made their students "tick”—see the relationship between classroom 
behavior and the students’ life outside school. Secondly, these teachers believed that 
classroom discipline was less of a problem if students knew the teacher was in 
contact with their parents and that classrooom discipline would be reinforced in the 
home. This reinforcement, however, was not confined to disciplinary matters; for 
example, one teacher used letters to parents and an extraordinary number of home 
visits to encourage parents to read with their children. By the end of the school year, 
one-third of the parents of this teacher’s students were reading regularly with their 
children. 

As these examples illustrate, the successful teachers were willing to expend 
considerable effort, and the results showed up in the classroom atmosphere they 
created. Their classrooms were well organized and suffered few disruptions. Stu¬ 
dents knew what was expected of them, and knew the purpose of each instructional 
unit. Above all, these teachers were able to challenge and motivate their students 
by achieving a balance between positive reinforcement and strict discipline. 


READING PROGRAM IMPLEMENTATION 

Another set of significant variables were related to program implementation. 
One important factor was the use of school resources for teacher in-service training. 
The schools in our sample varied in the amount and kind of staff in-service they 
provided. One school, however, devised a plan that provided consistent, ongoing staff 
development in a fairly inexpensive way. The school hired a physical education 
coach who works with all classes at each grade-level for one period a day. During 
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this time, the teachers at a given grade-level attend staff development sessions 
conducted by the school’s math and reading coordinators. These sessions are orient¬ 
ed to topics and problems that the teachers themselves raise. This type of staff 
in-service has the advantage of providing regular and practical staff development as 
well as physical education instruction for students, all at a relatively low cost. (Of 
course, we found that other programs of staff development also had positive effects 
on student achievement.) 

Another aspect of program implementation is the extent of interaction among 
classroom teachers. We found that teachers’ informal consultations with other 
teachers about the reading program produced significant advances in students’ 
reading achievement. At some schools this consultation is done on a regular basis 
and on the teachers’ own initiative. At one school, for example, the 6th grade 
teachers agreed to meet during their lunch hour throughout the first year of the 
school’s Preferred Reading Program. They discussed common problems and ex¬ 
changed ideas about the modification and adaption of reading program materials. 
At other schools, interaction among faculty members was built into the schools’ 
in-service program. For example, one set of elementary schools that feed into the 
same junior high school take turns presenting a classroom demonstration of their 
reading program, followed by a workshop two weeks later. At least one teacher from 
each school in the complex attends and then reports back to his or her school. This 
type of program allows teachers to see how reading instruction is conducted in other 
schools and, at the same time, facilitates teacher interaction both with a given school 
and across a number of schools. 

. Our research also indicates that the more teachers are encouraged to adapt or 
modify the reading program, the greater is the increase in their students’ reading 
achievement. Teachers modified their schools’ reading programs in a variety of 
ways. For example, a number of teachers prepared written exercises to correspond 
to a reading program’s oral drills, while others used supplementary books and 
magazines to apply reading lessons in ways relevant to student experience and 
interest. One aspect of program adaption that several teachers considered important 
was their effort to get interesting books into the students’ homes. Although there 
was always the chance that some books might not be returned to the school, these 
teachers believed the payoff from students reading at home, either alone or with 
their parents, was well worth any lost books. 

Our study concluded that, at least with regard to the schools in our sample, no 
prepackaged reading program or technology will automatically produce effective 
results. The significant factor is the manner in which individual schools and teach¬ 
ers adapt a particular program to their students’ needs. We did find, however, that 
individualized and diagnostic instruction as a classroom behavior positively affects 
reading achievement. But again, individualized instruction is a useful technique 
only to the extent that teachers understand the underlying basis of individualization 
and actually carry it out effectively. For example, we observed several classrooms 
using DRP (Development Reading Program), a program designed as individualized 
instruction. In some of these classrooms, however, the teachers essentially did no 
more than go through the motions, dispensing a variety of materials and supposedly 
dividing students into skill-level groupings; it was clear that these teachers did not 
understand the rationale of the program and were not making adequate distinctions 
among the skill-levels of their students. On the other hand, we observed other 
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classrooms where the reading program was primarily oriented toward basal readers, 
but the teachers were aware of differences among their students, devised individual¬ 
ized materials, and could target their instruction to specific skill-level groupings. 


CONCLUSIONS 

As noted elsewhere in this report, neither prepackaged reading techniques nor 
specific teacher background characteristics were among the factors found to contrib¬ 
ute to improved reading achievement. Instead, we found that policies enhancing the 
implementation of whatever reading program the school had selected were effective 
in raising students’ reading scores. The examples presented in this chapter show 
that individual principals and teachers can approach the task of reading instruction 
in a variety of ways. No school or classroom maximized all the factors that contrib¬ 
ute to reading achievement, but the schools and classrooms discussed in this chapter 
effectively used some of those factors to enhance their reading programs. In spite 
of the diversity of approaches and techniques we observed, however, one generaliza¬ 
tion can be made about the elements of a successful reading program: in practice, 
these factors reflect a belief on the part of principals and teachers that children can 
be taught to read, regardless of motivation or background, and that reading instruc¬ 
tion can be oriented to the needs of individual students. 




Chapter 5 

PARENT AND COMMUNITY INVOLVEMENT IN THE 

SCHOOLS 


The past ten years have witnessed increasing parent and community involve¬ 
ment in the schools. Community involvement is seen both as an end in itself, and 
as a way to affect educational outcomes. Most studies have concentrated on the 
former view; to date, no studies have demonstrated that community involvement of 
a certain kind over a prescribed time period affects educational outcomes. Problems 
encountered in obtaining comparable experimental and control populations, in 
measuring participation, and in specifying educational outcomes make such studies 
difficult. As a result, we are left with case-study information that leads us to believe 
that community involvement in a school is important without knowing exactly how 
it is important. 

In this small substudy, with its limited resources, we could not hope to go much 
beyond previous work in demonstrating the effects of parent and community in¬ 
volvement on educational outcomes—in this case, reading achievement. 

Nevertheless, we believed we could obtain a more complete picture or our 20 
schools if we analyzed some information about their communities. We therefore 
conducted a series of about 40 interviews with parents, community residents, agency 
personnel, and community organization people, collecting information about the 
role, function, and perceived effectiveness of parents and community members with¬ 
in the schools. 1 


METHODS 

A largely open-ended interview schedule was designed to assess the involvement 
and influence of parents and community groups with the schools, with particular 
emphasis on reading; and to determine the extent of openness of the school to the 
community, including activities undertaken by the school to encourage parent and 
community participation. We strove to make the instrument consistent with com¬ 
munity concerns. The instrument was reviewed by community members with 
experience in program evaluation, who assured the research team that it was both 
comprehensive and sensitive to community variables. 

Interviews were limited to a random subsample of ten schools from the original 
sample: five each from the Mexican American and Black communities. The schools 
selected reflected the spectrum of socioeconomic conditions to be found in those 
coummunities. Interviews were conducted by study team members who were ethni¬ 
cally identified with the respective minority communities included in the study. 


1 A Steering Committee, made up of representatives of each of the ethnic education commissions, 
advised the research team. This committee consisted of Mrs. Ruby Aguilar, Mexican American Education 
Commission; Mr. Walter Jones, Black Education Commission; Ms. Ann Mikol, American Indian Educa¬ 
tion Commission; and Mrs. Nancy Oda, Asian American Education Commission. 
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Respondents were sought from among parents’ groups (both those organized 
within the school and those outside), community groups who come in contact with 
the schools, and individual parents and community members known to be active in 
the schools or in school-related issues. In general, three respondents were inter¬ 
viewed from each school and several others from the community at large. Those 
interviewed included educational aides, student volunteers, members of community 
agencies, and district personnel who worked with the schools through the various 
area offices. 2 


LEVEL OF COMMUNITY INVOLVEMENT 

The interview data revealed a pattern of active school solicitation of parent and 
community involvement. However, certain kinds of parent/community efforts ap¬ 
peared to be much more effective than others. Large school-to-school differences 
were found in the amount of participation by community and parents. These differ¬ 
ences were consistent with the extent to which the school structured its relationship 
with parents and community to integrate them into the school and its activities. The 
level of involvement did not vary as a simple function of neighborhood stability and 
income level. One of the schools with a lower degree of participation and school/ 
community integration was in a visibly stable neighborhood of single-family homes, 
while one of the schools with - higher participation and integration level was in a 
neighborhood which was rapidly changing, and in which there were many multiple 
unit dwellings, generally crowded housing conditions, and high levels of transiency. 
It may be more difficult to obtain parent involvement in more transient communi¬ 
ties, but the key to such involvement appears to be the leadership both of school 
administrators and of concerned community residents. 

Based on the data obtained from the respondents, and observations of tangible 
physical and behavioral variables within the schools, a picture of community partici¬ 
pation was constructed that is represented by a continuum of school/community 
integration. Table 5.1 illustrates this continuum, with a range of points from column 
1 on the extreme left (a relatively low degree of integration of the community into 
the school) to column 5 on the extreme right (a relatively high degree of integration). 
Schools in each subsample were then ranked in terms of level of integration. 3 

The upper row of Table 5.1 represents the various policy stances taken by the 
schools relative to community involvement. Listed directly below them are exam¬ 
ples of activities schools may initiate to carry out those policies. 

Columns 1 and 2 represent relatively low levels of school/community integra¬ 
tion. The schools that fall into these categories appear to make outreach attempts 
in a traditional way. Typically, the school develops a program and asks parents to 
become involved. The midpoint of the continuum (column 3) represents the begin- 

2 Attempts were made to identify a spectrum of opinion in the schools. However, this was not always 
possible because of the problems presented by respondent identification and selection. Those identified 
by the school as active are most likely to be proponents. Dissidents, in general, are either not identified 
or no longer involved with the schools. This raises questions of response bias, which are common to many 
research efforts that attempt to determine "community” attitudes vis-a-vis some subject. It is also 
important to recognize that our analysis is based on a small sample of interviews at each school. 

3 The continuum represents only a convenient way to discuss the schools. Since schools upon which 
the continuum was based were then ranked in terms of the continuum, it should not be considered a test 
or validation of the approach. 
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Table 5.1 

A Continuum op School/Community Integration 


Hem 

1 

2 

3 

1 

. 5 

Policy positions 
taken by the 
schools relative 
to community 
involvement 

Asks for parent 
involvement 

Provides projects 
for parents 

Outreach programs 
that benefit commu- 
nily (legal, immi¬ 
gration. welfare, etc.) 

Space provided 
for parents in 
school 

Space with equip¬ 
ment or services 
useful to commu¬ 
nity apart from 
school needs 

Examples: 

Sends notes home 
(multilingual in 
non-English- 
speaking areas) 

Asks lor help 
in specific school 
projects (paper 
drive, candy sale, 
Cinco de Mayo fes¬ 
tival ) 

Classes and programs 
that have no connec¬ 
tion to the ongoing 
school program 
(special inservice 
programs) 

Parents know 
where they can 
use rooms. Of¬ 
fices or a special 
room for parents/ 
community 
provided 

Parents’ room equip¬ 
ped with sewing 
machines that can 
be used for personal 
needs 


ning of a broader approach to outreach; these schools provided programs and ser¬ 
vices for community people as well as parents. Additionally, the school may begin 
to have visibility in the neighborhood at community functions not directly related 
to the educational program. Columns 4 and 5 represent still higher levels of outreach 
and involvement, to the point that the school makes space available to parents and 
community people. The provision of space appears to foster a feeling of "belonging” 
among parents and community. 4 

The schools in our subsample were found on all points of this continuum. Almost 
all the schools had some very active parents—but the numbers of parents and 
community individuals those parents could in turn mobilize or delegate responsibil¬ 
ity to were much smaller at the schools with a lower degree of community involve¬ 
ment than at other schools. 

Issues and patterns of involvement varied between the Black and Mexican 
American subsamples. Different interviewers rated the schools within each subsam¬ 
ple; therefore, these findings will be presented separately. 


THE BLACK COMMUNITY 
Schools 

Parents at schools that exhibited low school/community involvement tended to 
be proud of the school, and expressed fewer concerns about both their own role in 
school decisionmaking and the competence of the school’s staff than was the case at 
the higher-involvement schools. Characteristically, they spoke of their schools as 
making certain things available to parents, and tended to rely heavily on the school 
administration for direction. Community involvement at these schools can best be 
characterized as passive. The schools’ relationship to the Parent Advisory Councils 
and P.T.A. was supportive and open, but these parents tended to see themselves and 


This provision of space (as in categories 4 and 5 on the scale) is modified in our ratings to take into 
consideration the contingencies of differing amounts of available space among schools. Thus, this category 
was defined as actual space allotted or parents’ and community members’ perceptions of freedom to move 
about in the space of the school. 
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to be seen by the school as tangential to school decisions and activities. Parents were 
not generally included in school decisionmaking councils. 

Column 3 in Table 5.1 represents a midpoint of school/community integration. 
Schools at this point and higher in the Black communities tended to have parents 
who had been active and involved with the school for long periods of time—say, over 
four years. These parents are knowledgeable about the progress of the school, and 
they are notably able to mobilize others. Although only a few parents may attend 
any given Advisory Council meeting, they know how to mobilize community interest 
when an important issue arises. Parents in these schools have begun to articulate 
their strategies for getting what they want for the school from the Board of Educa¬ 
tion. The column 3 school in the Black community sponsored a program of commun¬ 
ity events, and several parents were intent on bringing other community groups into 
contact with the schools. 

In the schools related to columns 4 and 5 in the Black community subsample, 
parents were in and around the building fairly continuously. They were included on 
planning and monitoring committees for all school events and activities, not merely 
those involving compensatory programs or auxiliary services. Although the parents 
at one school did not have their own room, they were familiar enough with the 
school’s physical plan to be able to find a room in which they could talk with our 
interviewers in privacy. Attendance at the average Advisory Council meeting was 
high—about 40 (compared with about 15 at column 1 and 2 schools). 

In the Black community school with the most parent/community integration, 
parents were organized even before the school was established. They were involved 
from its inception and influenced both the naming of the school and the naming of 
the Child Care Center. Children’s Centers in the Los Angeles Unified School District 
generally assume the names of the schools at which they are located, but in this case, 
the community residents strongly felt it should have its own name. This school is 
distinguished in other ways as well. Classes in sewing, millinery design, and typing 
are offered to community residents, and the parents have a conference room avail¬ 
able to them at all times. The Parent Advisory Council has subcommittees that 
monitor such disparate subjects as curriculum, employment of school staff, and 
physical plant maintenance. Attendance at the Parent Advisory Council meetings 
was reported to average around 50. 

Types of Parent and Community Involvement in the Black 
Communities 

Nonparents appear to be involved significantly only in the schools scoring mod¬ 
erate and high in community integration. In the most highly integrated schools 
(columns 4 and 5), there appears to be substantial involvement by students of local 
universities, as well as active involvement by nonparent neighborhood residents. In 
both schools there is a senior citizens group of volunteers, and in all three an open 
environment for volunteer activities. 

Involvement with reading was generally low in all schools. In most cases, the 
staff selected the reading program and sought approval of the Advisory Council. In 
some schools parents were asked to approve a fait accompli while in others alterna¬ 
tive programs were presented but the prior preference or selection of teachers was 
made clear. In only one school, the one ranking highest in community integration, 



45 


did it appear that the parents were objectively educated as to the merits and draw¬ 
backs of each program and allowed to make their own informed decision. Teachers 
participated in this process, so the two groups were essentially acting as equals in 
the decisionmaking process. 


THE MEXICAN AMERICAN COMMUNITY 
Schools 

The two schools in the Mexican American subsample at the lowest level (column 
1) of school/community integration attempted to interest parents by sending home 
notes encouraging the parents to visit the schools, but did not suggest specific roles 
for the parents. There was no school-wide policy on the role of parents in the 
classrooms; it was left up to the parent to discover whether a particular classroom 
teacher was receptive to his or her presence. As in the Black schools with low levels 
of community/school integration, parent participation was characterized by low 
levels of awareness of possible parent roles and school issues. Attendance at parent 
meetings was low, and the few active parents in the school seemed at a loss as to 
how to interest more parents. In one school, respondents felt the school was doing 
all it could to involve parents; in the other school there was a strong feeling that 
the school could be doing more. Neither of these schools enjoyed any significant level 
of involvement by community groups. 

Three of the five schools in the Mexican American subsample were relatively 
high in school/community integration (columns 3, 4 and 5, respectively). In these 
schools there was active participation by community agencies and groups and in¬ 
creased visibility of parents at the school. These schools provided services of a 
nonschool-related nature aimed at community concerns: workshops on immigration, 
legal services, and welfare rights, among others. These workshops brought in mem¬ 
bers of the community at large and created an image of the school as a community 
resource center. Two of these schools provided physical space for parents and parent 
projects. These parents’ rooms functioned as an integrative mechanism, actively 
involving those who might otherwise be reluctant to participate because they did not 
view such participation as appropriate, or because of embarrassment at not speak¬ 
ing English. The existence of on-going projects that used the special skills of the 
parents helped them to define their roles in the school. 

One school (column 5) had gone beyond providing space and parent projects. In 
this school, sewing machines had been purchased and parents were encouraged to 
use them for personal needs or to help make clothing for needy members of the 
community. 

While certain issues and cultural characteristics may bring together members 
of the Mexican American community, these same factors, if neglected, may create 
barriers to successful school-community relations. One example is language, often 
an obstacle in the school/community relationship. Many monolingual Spanish¬ 
speaking parents are active and make up a significant number of the parents on 
Parent Advisory Councils, yet they may not make informed decisions. Meetings are 
conducted in English and often are not translated because parents may be embar¬ 
rassed to admit that they do not understand English. Or, translation may only 
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outline the proceedings, and parents may not trust the translator. This problem is 
so pervasive that the level of awareness of monolingual Spanish-speaking parents 
can often be used as a gauge of the overall parent awareness in these schools. If 
monolingual Spanish-speaking parents are well apprised of the issues in their 
schools, it is likely that the school is making a sincere effort to communicate with 
the community and respond to its needs. 

Types of Parent and Community Involvement in 
the Mexican American Communities 

In the Mexican American schools, parents were involved as teacher helpers, 
classroom aides, and noon aides, and worked on special projects. In all of the schools, 
there was some paid parent training tied to government sponsored educational 
programs, which drew some parents into the school. They also participated by sitting 
on the Parent Advisory Council, P.T.A., or various special programs’ councils (e.g., 
Follow Through, Early Childhood Education). In one school, parents were invited to 
give workshops or classes on special skills they possessed. They sometimes func¬ 
tioned as representatives of the school at city-wide and district-wide meetings. In the 
schools with the highest levels of community integration, parents participated by 
availing themselves of services provided by the school. 

The involvement of nonparent, community-based groups in the schools was an 
important discriminator between schools with lower and higher levels of integra¬ 
tion. In the East Los Angeles area, community, high school, and university-affiliated 
groups have played a significant role in the schools. One organization based in the 
East Los Angeles area has been responsible for training community members in 
techniques for organizing parents in schools. Four of the five schools in the Mexican 
American subsample were recipients of the services of this organization. The schools 
that had parent workshops sponsored by the organization were those that ranked 
at the higher levels of community/school integration. 

An organization affiliated with a local university had presented weekly pro¬ 
grams aimed at ethnic identity and personal growth in several of the 6th grade 
classrooms in the three most highly integrated schools in the sample. 

While it is difficult to state whether high levels of school/community integration 
caused a school to have significant participation by community groups, or whether 
the higher levels of community participation was a cause of a strengthened school/ 
community relationship, it is important to note that the two factors correlated. The 
existence of nonparent, community-based groups in the schools was an important 
discriminator between low (column 1) and higher (column 3 and above) schools. 


FACTORS IMPORTANT IN SCHOOL COMMUNITY 
RELATIONS 

High levels of parent and community activity did not necessarily mean that 
parents were effective in setting and achieving goals. Those schools with low levels 
of school/community integration were characterized by few, if any, demands by the 
community. However, even in some schools with relatively high community involve¬ 
ment and activity, parents could not remember actually having made a demand on 
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the school, and there was relatively little awareness of how to go about making 
changes or expressing demands. In the schools where persons were most able to 
point to parent-initiated or community-initiated accomplishments, the respondents 
more readily related minor dissatisfactions and had a sense that they could create 
a climate of productive tension until they got what they wanted. There was a great 
deal of awareness in these schools (basically, 4 and 5 on the scale) of the administra¬ 
tive hierarchy, all the way up to the Board of Education. Several of the parents in 
these communities had dealt with school administrators at various levels in order 
to gain satisfaction on an issue of concern to the community. High parent/commun¬ 
ity efficacy in all the schools surveyed was characterized by a high level of awareness 
of school/community issues, familiarity with methods for obtaining satisfaction that 
transcended the immediate school setting, and the ability to generate productive 
tension within the school in order to ensure that parent/community concerns were 
acted upon. In schools where parent efficacy appeared to be low, tensions were 
played down and parents exhibited relatively little awareness of alternatives for 
effecting change. 

The role of the principal was repeatedly stressed as important in establishing 
a climate for school/community relations. In two schools, the principal’s leadership 
was cited as the impetus for community representation on school committees, in¬ 
cluding employment screening and the selection of the reading program to be used. 
In these schools, the principal took an active stance not only in involving the 
parents, but in educating them so that they could make informed choices. As a 
result, parents backed the principal to the hilt, even to the extent of going to the 
Board of Education to push for the favorable resolution of issues perceived to affect 
the interest of the school. In the schools characterized by active principal/commun¬ 
ity relations, parents appeared to regard the school as theirs, largely open to their 
input, and education was seen as the joint responsibility of the parents and school 
staff. In the schools with less active principal/community relations, there had been 
one or more changes of principal in recent years, and it was sometimes the case that 
the new principal was struggling to transform relative apathy into community 
involvement—or had difficulty in maintaining a previously established high level of 
commitment. 


COMMUNITY INVOLVEMENT AND READING GAINS 

While community involvement has intrinsic value, the specific relationship 
between community involvement and reading scores is also of concern. It does not 
necessarily follow that a high level of school involvement by both community and 
parents will influence reading achievement. To roughly assess the relationship be¬ 
tween parent involvement and student gain, we ordered schools in our community 
subsample according to both their perceived levels of community involvement and 
their percentile gains in reading achievement from grades 4 to 6, and compared the 
rank-orderings (see Table 5.2). These orderings suggest that for Black schools, levels 
of parent involvement appear to relate rather closely to student reading gains, an 
observation consistent with the results of our statistical analysis (Chap. 3), which 
found that high levels of teacher-parent contact were associated with reading gains. 
No such relationship appears in the Mexican American subsample, although it 
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Table 5.2 

Rank Ordering on Level of Community 
Involvement and Reading Gains" 


Black 

Community Subsample 

Mexican American 

Community Subsample 

School 

Community 

Involvement^ 

Reading 
Score Gain 

School 

Community 

Involvement 

Reading 
Score Gain 

B 

5 

+ 5 

S 

5 

-2 

E 

4 

(c) 

F 

4 

-18 

J 

3 

-3 

P 

3 , 

-7 

M 

2 

4 

C 

1.5* 

-5 

I 

1 

-11 

D 

1.5 d 

-6 


a Grades 4 to 6 mean percentile gains. 
^The highest rank is 5. 


c Gain scores at this school were judged invalid because of test exposure 
problems. Since this school was eliminated from the analysis in Chap. 2, 
we have eliminated the gain score from this table. However, the pattern 
of community variables for this school was consistent with the findings for 
other schools in the Black subsample. 

d Tied ranking. 


should be noted that the school that was most effective in that subsample, if not in 
producing reading gains, then in holding back losses, was also the school with the 
highest level of community/school integration and awareness among monolingual 
Spanish-speaking parents. This of course does not explain the absence of a relation¬ 
ship between reading scores and school/community integration rankings in the 
remaining schools in the sample. 

On the basis of the data we collected and our experience in related research, we 
can speculate on some possible reasons for the apparent relationship between parent 
and community involvement and student reading gains in the Black sample, and the 
lack of such a relationship in the Mexican American sample. We noted earlier that 
level of parent/community activity cannot be equated with efficacy in influencing 
reading achievement. This is particularly so in Mexican American communities. 
Language (and other factors) may be creating barriers to parent/community effec¬ 
tiveness; we may well be observing parallel processes in the classroom and commun¬ 
ity. In addition, the immediate needs of the community may at times be more salient 
than the specific concerns of classroom instruction. Outreach programs that offer 
services may increase the level of school/community integration and benefit partici¬ 
pants without strongly affecting educational programs or outcomes. 

Outreach programs in the Black communities that provide services are less 
likely to be the initial attraction for parent involvement in the schools. Such pro¬ 
grams increase the level of school/community integration and help to meet com¬ 
munity needs and concerns but are more likely to include an educational orienta¬ 
tion. 
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We say little about causality, of course. At the same time, the pattern of findings 
presents some implications for policy. It appears that in Black communities, level 
of involvement in the schools may be related to reading gains. No such relationship 
was found in the Mexican American communities. The mixed pattern of effect in our 
sample of Mexican American schools suggests the possibility of more complex rela¬ 
tionships that are obscured by the presence of variables we did not measure. It is 
the judgment of the interviewers that if community needs and processes were better 
understood, a relationship between level of involvement and gains might be found 
in Mexican American as well as Black communities. We believe that parent-com¬ 
munity involvement should continue to be strongly encouraged in both communities 
and that further efforts to clarify the relationships in the Mexican American com¬ 
munity should be considered. 



Chapter 6 
CONCLUSIONS 


OVERALL ASSESSMENT 

In general, our findings confirm the appropriateness of the current district 
reading program policies of school-level goal setting, teacher involvement in pro¬ 
gram planning, and the allocation of important decisionmaking authority to schools 
and teachers. We found that for both Black and Mexican American students, school 
and classroom inputs were directly responsible for significant changes in students’ 
reading achievement. In other words, school and classroom decisions matter for the 
education of the minority students we studied, over and above the importance of 
student background characteristics such as socioeconomic status and prior reading 
knowledge. 

We also identified specific classroom and school factors that were related to 
reading achievement in our sample of Black children, though the strength of these 
findings depends on assumptions about the validity of test results arising from 
evidence of test exposure. We were unable to identify specific school or programmat¬ 
ic factors related to the real gains we discovered for Mexican American children, 
even though the evidence is clear that school and classroom inputs do in fact matter 
for the Mexican American sample. 

Our limited analysis of the effects of parent and community involvement on 
education leads us to believe that such involvement should be encouraged. Although 
our findings are only suggestive, it appears that vigorous school efforts to involve the 
community are often associated with improved outcomes for children’s education. 

'The following discussion details this overall assessment by summarizing the 
specific findings for our sample. We then consider the applicability of these findings 
to other schools and to future policies. 


SUMMARY OF FINDINGS 
Reliability of Test Results 

Reliability analysis is performed to assess whether test results are internally 
consistent and reasonably free from random errors and other inaccuracies. We 
found solid evidence in our data that the reading tests used by the LAUSD meet 
established reliability standards. Reliabilities were high for both the large sample 
of student records we analyzed, and for each school. 

Validity of Test Results 

Our analysis of test validity is relevant both to the interpretation of our findings 
on effective school policies and to the district’s methods of assessing school-by-school 
changes in reading achievement. Our first point concerns the way that reading gains 
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are measured. The current policy is to report cross-sectional change, that is, to 
compare different groups of students from one year to the next. But year-to-year 
differences can be largely due to differences in the ability and composition of succes¬ 
sive groups of students and therefore are not a truly valid measure of improvement. 
To test this, we compared changes in successive 6th grade median scores (the dis¬ 
trict’s measure) with changes in individual children’s scores over time, and found 
only a moderate rank order correlation. To assess true change, we must compare the 
performance of the same group of students between, say, 1974 and 1975 as they 
passed through the 5th and 6th grades. For that reason, we recommend that the 
district consider the use of longitudinal cohort analysis like the one performed for 
this study. This involves collecting and reporting data on the progress of individual 
students over time, rather than grade level data on successive cohorts. 

Our second set of comments concerns test preparation. Used appropriately, test 
preparation enhances the validity of test scores and allows meaningful cross-school 
and cross-classroom comparisons. However, overzealous test preparation can pro¬ 
duce invalid scores and mask failures to educate. Our findings indicate that the level 
and range of test preparation activity varied widely. Many teachers reported consid¬ 
erable pressure to show test score gains, and few saw much utility in test scores as 
indicators of students’ performance. This combination creates an environment that 
is ripe for overzealous preparation. Exposure of children to tests or at least intensive 
drilling on material likely to be covered on upcoming tests can—and we believe 
sometimes did—occur in a few classrooms. We emphasize, however, that for about 
90 percent of the teachers we surveyed no such questions as to the validity of 
measured gains are warranted. 

Effect of School and Classroom Factors on Reading Achievement 

Students who had similar backgrounds and prior achievement levels, but who 
had different school and classroom experiences, achieved different rates of reading 
progress. That is, some schools and some classrooms within a school contributed 
more to their student’s achievement than did others. This result held for both 
Mexican American and Black children, and did not change when we excluded test 
outcome data that may have been spurious as a result of test exposure. 

We infer that school and classroom effects on reading progress were strong 
because very different kinds of influences—school policies and classroom settings— 
were received by children depending on where they were enrolled. This finding lends 
support to district policy, which grants authority and resources to local schools to 
design and operate reading programs. 

We also found that at both predominantly Black and Mexican American schools, 
principals can make highly accurate assessments of the instructional ability of their 
teachers. This suggests that principals are indeed an appropriate locus for school- 
level instructional policy decisions, at least in our 20 schools. 

As we discuss below, there are no fixed programs or single strategies that can 
raise reading scores all at once. But it does appear that reading instruction can be 
improved on a school-by-school and classroom-by-classroom basis when individual 
programs are carefully planned, adapted to local circumstances, and implemented 
effectively. 
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Factors Affecting Reading Achievement for Black Children 

On the basis of our analysis we conclude that three broad categories of factors 
produce improved reading achievement for Black children: program content, im¬ 
plementation strategies, and classroom atmosphere. These relationships were strong, 
and remained consistent even when we excluded test scores for classrooms where 
there was strong evidence of prior test exposure. When we further excluded test 
scores for additional classrooms showing unusually large gains, some of the associa¬ 
tions we report here became weaker. Confidence in the soundness of our results must 
therefore be tempered by the reader’s judgment about how much test exposure 
actually occurred. We believe that the middle-ground position we used in generating 
the results we report here is at least reasonable. 

We found that reading achievement improved when the reading program used 
varied materials for different students at the same time. Other more novel ap¬ 
proaches to instruction, such as open classrooms and team teaching, were not consis¬ 
tently related to measured gains on standardized reading tests. Our analysis of other 
program content measures also supported the conclusion that reading curricula do 
not, by themselves, determine the success or failure of a school’s reading program. In 
fact, there is wide variation in average student progress among classrooms that have 
very similar curricula and reading strategies. This may imply that any of the 
reading programs we studied can be effective depending on how well it is implement¬ 
ed and taught. Principals may wish to consider whether an effective program of 
individualization—such as the one measured by our "use of varied materials” ques¬ 
tion—can be appropriately used in their schools, as well as the question of how such 
a program could be implemented. Additional teacher training, and visits by resource 
staff to individual classrooms to observe and suggest activities that will strengthen 
teachers’ use of individualized techniques, may be useful in pursuing this goal. 

We found that program implementation affects reading achievement across a 
range of instructional methods and school settings. The implementation strategies 
chosen by a school to support its reading program are crucial, because how a reading 
program is carried out strongly affects how much students learn. The importance 
of implementation strategies is very great even after taking into account the impact 
of curriculum, classroom setting, and teacher attributes. Successful implementation 
strategies in our sample included adequate teacher training (especially with respect 
to the use of varied materials), a high level of informal consultation among staff, and 
freedom on the part of teachers to make modifications in the reading program that 
increase its pertinence to the practical situations they encounter in their class¬ 
rooms. In general, the more active was the role of teachers in implementing the 
reading program, the more reading achievement improved. 

Classroom atmosphere factors that affected reading included an absence of dis¬ 
ruption, frequent contact between teachers and parents, and a feeling of efficacy on 
the part of teachers. Our measurements of the classroom setting are not subtle, but 
they confirm the hypothesis that the background for learning can affect teachers’ 
and students’ productivity. Moreover, we found that teachers matter for reading: 
their sense of being able to "get through” to students, their commitment and morale, 
help to determine how much children learn. We were surprised to find little or no 
association between teachers’ background attributes and reading progress in their 
classrooms. Years of experience, ethnicity, college major, or place where the under¬ 
graduate degree was obtained appeared to have little influence. 
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Policies that strengthen teachers’ commitment and provide them with the skills 
and resources necessary for them to perform well in difficult circumstances may 
contribute to improved classroom settings, and thus to improved reading achieve¬ 
ment. 

As an illustration of the quantitative importance of the various specific factors 
that we have identified as important, we estimate that exposing the average student 
in our sample to all of the approaches found to be effective would have resulted in 
a 10 to 20 point percentile gain for that student over the course of the 6th grade, 
instead of the decline of 3 percentile points that actually occurred. 

As we stated earlier, our analysis demonstrated that principals’ assessments of 
teachers accurately predict how much students will gain in reading. While it is 
likely that parents and teachers also have valuable information about school prob¬ 
lems, the principal has a unique opportunity to observe the whole situation at his 
or her school, and to act to improve it. Principals can exercise their leadership in 
a number of ways that may help produce reading gains, by observing teachers and 
making specific recommendations for change. Principals can facilitate successful 
program implementation by working to provide adequate training for teachers, and 
they can encourage teachers to consult with each other and to modify their pro¬ 
grams to meet their students’ needs more fully. 

For the predominantly Black schools in our sample, high levels of parent and 
community involvement appeared to be associated with better reading outcomes. 
Although our study of this relationship was constrained by resource limitations, and 
although no similar finding was derived for Mexican American schools, we suggest 
that the current district policy is sound in encouraging schools to involve the com¬ 
munity in the educational process. 

In summing up our substantive findings, it is important to emphasize that no 
single school or classroom factor , taken by itselfis likely to produce large increases 
in reading achievement We found many school and classroom factors that were 
strongly related to reading performance, but no one factor or small group of factors 
dominated our findings. We believe this is to be expected. Reading instruction is far 
too complex to allow for simple policies or "quick fixes.” 

Our findings suggest that continuation of the district’s policy of school-level 
goal-setting for reading, and delegation of authority to schools and teachers to make 
program planning and implementation decisions, is appropriate for the sample of 
selected schools. 


APPLICABILITY OF THE FINDINGS TO OTHER SCHOOLS 
AND FUTURE POLICIES 

In considering the degree to which our findings may be applicable to other 
educational settings, we must repeat the caveats we have stressed in this report: we 
worked with a nonrandom sample of predominantly Black and Mexican American 
schools in Los Angeles, and studied intensively only what occurred in the 6th grade 
in those schools. Consequently, generalizations to other minority schools, to 
predominantly Anglo schools, to other districts, and to other grade levels can be only 
suggestive. 

With those caveats in mind, we offer the following policies as general guidelines: 
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• Because our analysis illuminated the importance of particular schools and 
classrooms within schools for reading progress, a good measure of local 
school autonomy seems warranted, particularly in large districts. This 
suggestion is bolstered by the conformity of our results to previous research 
in the field and by our finding that principals we queried have accurate 
knowledge of their teachers’ relative effectiveness in the classroom. 

• Though the specific factors we identified as efficacious were associated only 
with reading gains for Black students, they too are confirmed by other 
research on both majority and minority schools. Teacher flexibility, in¬ 
dividualized approaches, teacher-to-teacher consultation, training and 
morale enhancement for teachers, and involvement of parents and com¬ 
munity members in the educational process all appear to have payoffs— 
and all point again toward the desirability of school-level decisionmaking. 

• Therefore, the national trend we observe toward central direction in school 
districts may not be very productive, at least for large, complex school 
systems. 

To the extent permitted by local resources, school district research staff should 
attempt to replicate and extend the findings of this limited study, and should pursue 
issues that we were unable to resolve adequately. These include: the utility of 
standardized tests for gauging reading achievement, the relevance of national norm 
criteria to minority children, the consequences of the growing emphasis on test 
scores as measures of success, and the factors associated with true reading achieve¬ 
ment for students from non-English-speaking backgrounds. Further research of this 
kind is likely to provide useful information for policy decisions. 



Appendix A 

SAMPLE DESIGN AND DATA COLLECTION 


This appendix describes various technical aspects of the study performed. We 
first outline the procedures used to select the elementary schools included in our 
sample. We then discuss how we traced students from the 6th grade classes of 1974 
and 1975 to the junior high schools they attended in February and March 1976 
where their cumulative records were lodged, and the extent to which the sample of 
students for whom data were obtained are representative of the population of 6th 
grade students in the selected elementary schools. We also describe the methods we 
used to collect data on school and classroom factors. 


SELECTION OF THE SAMPLE ELEMENTARY SCHOOLS 
Criteria Used 

The criteria used in selecting the 20 sample schools were that each school should 
demonstrate substantial or consistent test score gains, a predominantly minority 
enrollment, and enough 6th grade classrooms to exhibit variation in reading ap¬ 
proach, and that each should serve a population that is low in socioeconomic status. 

The LAUSD’s Request for Proposal criteria for the sample schools were that 
each should have a predominantly minority enrollment and substantial and consis¬ 
tent test score gains on the 6th grade reading test. These criteria relate to the 
purpose of the study, which was to identify those elements of the reading program 
that improved reading achievement in minority schools. 

To further define an appropriate population, Rand added two more criteria. 
Some schools are extremely small (for example, Solano), with only one or two class¬ 
rooms at each grade level; the school-size criterion was formulated to ensure a 
sufficient sample of students from each school so that we could measure variation 
in school-determined aspects of the reading program. In addition, a few predomi¬ 
nantly minority schools serve populations of relatively high socioeconomic status 
(for example, Windsor Hills). Because children there were expected to outperform 
minority children elsewhere, we added a criterion for socioeconomic status. 

To quantify these criteria, we used the following definitions: 

Substantial Gains: An increase in the 6th grade class total reading test score 
median (national norm percentile for the school) of 15 or more percentile points from 
Fall 1972 to Spring 1975. Schools with substantial gains were not required to have 
consistent gains. 

Consistent Gains: No gain in any one intermediate year greater than two-thirds 
of the total gain. Fall 1972 to Spring 1975. 

Minority Enrollment: At least 60 percent nonwhite enrollment, Fall 1972, and 
at least 80 percent nonwhite enrollment, Fall 1974. (Nonwhite enrollment includes 
Black, Mexican American, Native American, Asian American, Filipino, and other.) 
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No school was to be selected that derived more than one-half its minority enrollment 
from the Asian American population. (This limitation was included because Asian 
Americans normally have higher achievement levels.) 

School Size: Enrollment of at least 400. 

Socioeconomic Status: Title I rank of 200 or lower. Of the district’s 436 elemen¬ 
tary schools, ranks 1 to 200 represent the "most disadvantaged” schools. The Title 
I rank is based on AFDC (Aid to Families with Dependent Children), free lunch 
program participation, the distribution of family income in the 1970 Census of 
Population, and the distribution of assessed values for single-family residential 
properties. 

To select the schools, we first identified the 200 that qualified on the basis of 
socioeconomic status. We then reduced this list by the school-size and minority 
enrollment criteria, which left 87 schools for consideration. The test-score changes 
(5th grade to 6th grade) were examined for all of these schools, and the 8 schools with 
substantial gains (15 or more percentile points) were selected for the sample. (These 
schools did not necessarily have consistent test score changes.) We also identified 19 
schools that registered consistent test-score changes, and divided them into two 
groups: those with less than a 10 percentile point gain and those with gains of 
between 10 and 14 percentile points. We selected 6 schools from each group, so as 
to maintain a balance in the sample between predominantly Black and predomi¬ 
nantly Mexican American schools. Table A.l presents the sample schools and their 
values cn each of the selection criteria. 

Comparison of the Sample Schools with All LAUSD Schools 

In comparison with all LAUSD schools, the 20 sample schools have a higher 
percentage of minority students, larger size, lower socioeconomic status, and larger 
gains on 6th grade reading test scores. (See Table A.2.) 

Although the sample schools were constrained by study objectives to have at 
least 80 percent minority enrollment, the ratio between Black and Mexican Ameri¬ 
can students in the sample is the same as their ratio in the district. Asian American 
and Native American students, however, are not represented in the sample in 
proportion to their ethnic distribution in the school district. The two schools with 
the highest proportions of Asian Americans were excluded from the sample. Caste- 
lar was excluded because its enrollment was more than 50 percent Asian American; 
Coliseum Street, which has an Asian American enrollment of 46 percent, was ex¬ 
cluded because its Title I rank was greater than 200. The two schools with the largest 
shares of Native Americans (Park and Woodlawn) have more than 20 percent non¬ 
minority enrollments. 

Because the sample schools had to have at least 400 students, they are 70 percent 
larger on average than other district schools. There were 146 schools in the district 
smaller than the smallest school (enrollment of 591) in the sample. 

The district-wide gain in median national norm percentiles on the reading test 
for 6th grade between 1972 and 1975 was 5 percentile points, or 15 percent of the 
base-year score. The sample schools, which had a considerably lower mean score in 
1972, had increased their scores during the period of the study to the extent that 
their mean score was closer to the district median by 1975. 

Sample schools were lower in socioeconomic status than the mean for the dis- 
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Table A.2 

Comparison of Sample Schools with All LAUSD Schools 


Characteristic 

AH Schools 

Sample Schools 

AFDC Rank, 1974 

Range 

1-436 

11-159 

Mean 

218 

75 

Ethnic Distribution, 

Fall 1974 (%) 

Black 

24.8 

41.7 

Mexican American 

31.8 

53.1 

Asian American 

4.9 

2.7 

Native American 

0.3 

0.1 

Other 

38.2 

2.4 

Reading test scores, 

CTBS-Q2, Grade 6 a 

1972 

33 

15 

1973 

33 

18 

1974 

38 

22 

1975 

38 

31 

Gain, 1972-75 

5 

16 

Enrollment, fall 1974 

Range 

146-1641 

591-1462 

Mean 

736 

1030 


a Median national norm percentile for the district, mean of the 
median national norm percentiles for the sample schools. A med¬ 
ian is published for the entire district and for each school for each 
year. Without the raw data, it is impossible to construct medians 
for groups of schools; therefore, the mean of the medians for the 
sample schools is used. 


trict. This means that the typical sample school had a Title I rank closer to the most 
"disadvantaged” and lowest in socioeconomic status than that of the typical school 
in the district. 


Comparison with Minority Schools in the LAUSD 

The 20 schools had lower median national norm percentile reading test scores 
in 1972 than the average school with at least 80 percent minority enrollment. They 
also had greater gains than the average minority schools, and in 1975 had higher 
test scores. (See Table A.3.) During this period the 80-percent-minority schools had 
gains in median national norm percentiles higher than the district’s gain for all 
students. 

The sample schools were larger on the average than the typical minority school 
in the district, and had slightly more Mexican American and fewer Black, Asian, 
and other ethnic students enrolled than the average minority school. The sample 
schools had a slightly lower Title I rank than the average minority school in the 
district. This means that sample schools were slightly lower in socioeconomic status 
than the average minority school. 
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Table A.3 


Comparison of Sample Schools with All 
80-Percent Minority Schools in the LAUSD 


Characteristic 

All 80-Percent 
Minority Schools 

Sample Schools 

AFDC Rank (1974) 

Range 

1-396 

11-159 

Mean 

100 

75 

Ethnic Distribution, 

Fall 1974 (%) 

Black 

50.7 

41.7 

Mexican American 

41.9 

53.1 

Asian American 

3.9 

2.7 

Native American 

0.1 

0.1 

Other 

3.4 

2.4 

Reading Test Scores, 

CTBS-Q2, Grade 6 a 

1972 

18 

15 

1973 

19 

18 

1974 

23 

22 

1975 

28 

31 

Gain, 1972-1975 

10 

16 

Enrollment, Fall 1974 

Range 

146-1641 

591-1462 

Mean 

851 

1030 


a Mean of school mediae national norm percentile. A median is 
published each year for the district as a whole and for each school 
in the district. Without the raw data, it is impossible to contruct 
medians for groups of schools; therefore, the mean of the school 
medians is used. 


DATA COLLECTION 
Student Cumulative Records 

To collect historical data on the 1974 and 1975 graduates of the 20 schools, it 
was necessary to locate their elementary cumulative record folders at the students’ 
current junior high schools. At the completion of 6th grade, the elementary school 
records are sent to the junior high school where the student is expected to enroll the 
following year. If the student does not, the records may be returned to the elemen¬ 
tary school, or sent to another junior high school that requests the records, or filed 
in an inactive file. Records are tsent to other California school districts when request¬ 
ed by the receiving schools for transferring students. 

To identify the junior high schools that the graduates of our 20 schools would 
be expected to attend, we used the Contributing Schools Report (prepared each fall 
by the Maps and Boundaries Section, Educational Housing Branch, School Building 
Planning Division) and the Permits with Transportation (PWT) Statistical Informa¬ 
tion Report (prepared by the Permits with Transportation Office). The Contributing 
Schools Report identifies the elementary schools that contribute to each junior high 
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school. The junior high schools identified as receiving schools for our 20 schools are 
listed in Table A.4. Most of these schools are sending schools in the Permits with 
Transportation Program (PWT); that is, some of their students are transferred to 
other junior high schools. (Table A.5 is a list of the receiving PWT junior high schools 
for each sending junior high school in Table A.4.) The Permits with Transportation 
Program was designed to relieve crowding in inner city schools and to meet earth¬ 
quake safety standards by transferring students by bus to less crowded, new schools. 

The junior high schools listed in the Contributing Schools Report were reviewed 
to ensure that a substantial proportion of each graduating class could be expected 
to be found. After data collection activities were conducted at each junior high 
school, the number of records found from each elementary school was compared 
with the number expected. If the number of records found was significantly less than 
expected, the PWT schools for that junior high school were checked and added to 
the data collection schedule if the school had enrolled a reasonable number of 
students from the elementary school in question. This process continued throughout 
the period of data collection activity to ensure that an adequate number of cases 
from each school were located. Table A.6 lists 6th grade sizes and numbers of cases 
for the 20 sample schools. 

We identified 37 receiving junior high schools where graduates of the 20 sample 
schools were expected to matriculate. We collected data at 32 of these schools, 


Table A.4 

Receiving Junior High Schools for 
Sample Elementary Schools 


School Name 

School Name 

Adams 

Hughes 3 ’ 0 

Audubon 

King 

Belvedere 

Maclay 

Berendo 

Mann 

Bethune 

Markham 

Burbank 3 

Mount Vernon 

Burroughs . 

Muir 

Carver 

Nightingale 

Clay 

Nobel a > c 

Drew 

Pacoima 

Edison 

Pasteur 

El Sereno 

Portola c 

F oshay 

Revere 0 

Fulton 3,c 

San Fernando 

Gage c 

Sepulveda a>c 

Gompers 

Stevenson 

Griffith 

Virgil 

Harte 

Hollenbeck 

Webster^ 


a Receiving junior high school, class of 1975 
only. 

^Receiving junior high school, class of 1974 
only. 

c Not sending school in the PWT program. 

J 

u Both sending and receiving school in PWT 
program. 
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Table A.5 

"Permits with Transportation” Junior High Schools, 1975 


Sending Receiving No. of 

School School Pupils 


Adams Fulton 1 

Holmes 2 

Revere 2 

Audubon Bancroft 2 

Columbus 3 

Emerson 40 

Henry 3 

Holmes 3 

Hughes 2 

Madison 6 

Millikan 2 

Nobel 5 

Palms 3 

Revere 86 

Sutter 3 

Belvedere Portola 37 

Berendo Bancroft 1 

Holmes 3 

Hughes 1 

Reed 115 

Sequoia 14 

Bethune Emerson 2 

Fulton 6 

Holmes 2 

Hughes 1 

Madison 3 

Nobel 2 

Parkman 1 

Sepulveda 14 

Sequoia 6 

Burbank Millikan 2 

Burroughs Bancroft 1 

Emerson 2 

Madison 1 

Palms 2 

Revere 1 

Carver Bancroft 16 

Columbus 1 

Emerson 12 

Fulton 2 

Henry 1 

Madison 3 

Revere 29 

Sepulveda 13 

Clay Carnegie 14 

Emerson 4 

Fleming 2 


Sending Receiving No. of 

School School Pupils 

Fulton 1 

Holmes 2 

Hughes 15 

Madison 8 

North ridge 4 

Parkman 6 

Porter 4 

Revere 2 

Sepulveda 4 

Sequoia 1 

Sutter 1 

Van Nuys 3 

White 4 

Wright 3 

Drew Madison 5 

Mulholland 1 

Parkman 3 

Revere 2 

Edison Bancroft 5 

Emerson 9 

Madison 7 

Miiiikan 1 

Mulholland 1 

Parkman 1 

Revere 5 

El Sereno Millikan 3 

Portola 21 

Foshay Carnegie 7 

Columbus 7 

Emerson 9 

Henry 3 

Holmes 8 

Hughes 1 

Madison 1 

Millikan 2 

Nobel 2 

Northridge 2 

Porter 36 

Revere 2 

Sequoia 7 

Gompers Dana 8 

Emerson 2 

Fleming 15 

Madison 4 

Revere 6 

Sepulveda 4 

White 14 

Griffith Portola 70 
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Table A.5 (Continued) 
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Table A.6 

Size of 6th Grade by School and Cases in Study, 
Classes of 1974 and 1975 




1974 



1975 


School 

Size of 

6th Grade 

Cases 

% of 
Class 

Size of 
6th Grade 

Cases 

% of 
Class 

Alta Loma 

125 

82 

66 

110 

88 

80 

Angeles Mesa 

152 

104 

68 

117 

89 

76 

Ascot Avenue 

114 

90 

79 

103 

93 

90 

Dacotah Street 

113 

80 

71 

98 

76 

78 

Eastman Avenue 

176 

146 

83 

171 

152 

89 

Ford Boulevard 

153 

109 

71 

99 

75 

76 

Harrison 

86 

66 

77 

74 

65 

88 

Hillside 

72 

58 

80 

77 

61 

79 

Manhattan Place 

148 

92 

62 

164 

119 

72 

McKinley Avenue 

115 

95 

83 

106 

72 

68 

Miller 

153 

113 

74 

114 

88 

77 

118th Street 

93 

60 

64 

90 

71 

79 

112th Street 

109 

76 

70 

89 

71 

80 

Pacoima 

189 

121 

64 

183 

154 

84 

Rosemont Avenue 

144 

88 

61 

121 

103 

85 

Rowan Avenue 

162 

146 

90 

179 

146 

82 

Sierra Park 

157 

141 

90 

163 

145 

89 

10th Street 

146 

116 

79 

138 

125 

90 

28th Street 

133 

93 

70 

122 

95 

78 

Vermont Avenue 

126 

110 

87 

124 

102 

82 

total 

2666 

2666 

74 

2442 

1990 

81 


including 4 that received students in the PWT program. The aim was to find records 
for at least 80 percent of each graduating class. Initially, 27 schools were scheduled 
for data collection; 5 more, were scheduled to increase the proportion of records 
found from the 20 sample schools. Table A.7 describes the kinds of information 
collected in the 20 schools; Table A.8 and A.9 present the number of records located 
at the junior high schools for the Classes of 1974 and 1975. 

The data items recorded on prepared coding forms from the cumulative record 
files located in the junior high schools were: 

• Elementary school attended 

• Student’s name (for coding verification; student identification by name has 
not been retained in our data files) 

• Sex 

• Ethnic background (as determined from name, birthplace, parent’s photo, 
and language spoken in the home) 

• Presence of parents in the home (from listing of guardians, step-parents, 
comparison of student’s and parents’ names and person responsible) 

• Parents’ occupations 

• 1st grade reading test score (Cooperative Primary) (Class of 1975 only) 

• 3d grade reading test score (Stanford or Cooperative Primary) 

• 4th grade Spring reading test score (CTBS-Q2 or R2) 
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Table A.7 

Descriptions of Data Collected in the 20 Elementary Schools 


Information 

Source of Information 

Examples 

School Data Sheet 

Principal/Office Manager 

Enrollment, turnover, specialists, 
aides 

Principal interview 

Principal 

Activities to support program im¬ 
plementation; teacher reactions to 
program; community involvement; 
reasons for reading gains; problems 
encountered in implementation 

Principal assessment 
of teacher skills 

Principal 

Use of management system; combining 
teaching of skills and comprehension; • 
maintaining reasonable discipline 

Reading Coordinator 
interview 

Reading Coordinator 
or equivalent 

Components of reading program; 
successful instructional techniques; 
community involvement; problems 
encountered in implementation 

Teacher questionnaire 

6th grade teachers, 

1974-75 

Methods of reading instruction; 
emphasis on test-taking skills; 
training received; resources, 
aides, parent involvement; support 
from staff, specialists, principal; 
problems encountered; results of 
reading program 

Checklist for 6th graders 

6th grade teachers, 

1974-75 

Special services received; pan ..t 
contact; motivation 


• 5th grade Spring reading test score (CTBS-Q2 or R2) 

• 6th grade Spring reading test score (CTBS-Q2) 

• 6th grade Spring math test score (CTBS-Q2) (Class of 1975 only) 

• 6th grade Spring reading test score (CTBS-S2) (Class of 1975 only) 

• Date of continuous enrollment in sample school 

• 6th grade teacher’s name 

• Days absent, 4th, 5th, and 6th grades 

• Names of two 6th grade reading textbooks (Class of 1975 only) 

• Number of reading textbooks listed (Class of 1975 only) 

Information on the junior high school cumulative record folders was used only 
where photographs were needed for ethnic identification or for clarification of infor¬ 
mation on the elementary folder. 

Two teams of specially trained coders recorded the information. They worked 
at the junior high schools under arrangements made with the principal, head coun¬ 
selor, and credit clerk. The coders searched the 7th and 8th grade files for records 
of graduates of any one of the 20 schools. Because the junior high schools organize 
their cumulative record files in various ways, the teams checked any special files 
(Title I, Mentally Gifted Minors, English as a Second Language, Educable Mentally 
Retarded) and inactive files so as to locate all types of students from each of the 
sample schools. 
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Test score data for the Class of 1975 from the sample schools were also obtained 
for 4th, 5th, and 6th grades using computer-printed test scores for students by 
school. To acquire information on the 6th grade test results for each school, and to 
compute changes for each school, it was necessary to do a hand match of the comput¬ 
er records with the information coded from the cumulative records. (Test scores for 
each grade were not on the cumulative record files for each year for each school.) 
For grades 4 and 5, these printouts were obtained from the Title I Research and 
Evaluation Office, and for grade 6 from the Research and Evaluation Office of the 
District. These printouts were matched by name and birthdate to information col¬ 
lected on the cumulative record coding form. The name was not included in any 
computerized records created at Rand, so this matching was carried out by clerks 
on the individual records. Added to the basic data records were 540 grade 4 scores, 
399 grade 5 scores, and 205 grade 6 scores. 

Representativeness of the Sample 

To ascertain whether the sample of students for whom we found cumulative 
records is representative of all students who took the 6th grade reading test (CTBS- 
Q2), we made a comparison of test results for the graduating Class of 1975. This 
comparison was made possible by hand-matching the 6th grade computer printout 
of the CTBS-Q2 results to the data recorded from the cumulative record folders, to 
ensure there was complete information on the 6th grade scores. 

The sample of the Class of 1975 included 76 percent of all test-takers in the 20 
schools. The mean raw score on the CTBS-Q2 was 54 for both the sample and all 
test-takers. At the school level, nine of the schools had exactly the same mean raw 
score in the sample and for all test-takers. In eight schools the difference was only 
1 between the mean raw scores of the sample and all test-takers. In two schools 
(Dacotah Street and Pacoima) the difference in mean raw scores was 2, and in one 
school (118th Street) the difference was 3. Thus we have strong evidence that the 
students for whom it was possible to build longitudinal records of test-score results 
constituted a representative sample of all test-takers in the student population. 

Collection of Data on School and Classroom Factors 

We conducted two kinds of studies in our 20 schools. We wanted, first, to get a 
full picture of how reading programs fit into the larger world of the school. Second, 
we wanted to conduct a comprehensive survey of reading instruction at each of the 
schools. We began by studying three schools in detail, to ascertain the kinds of 
differences among reading programs that should merit our attention in all of the 
sample schools. We also tried to discover how schools act as the setting for a reading 
program—for example, how special programs such as Title I affect reading, how a 
new math curriculum might consume more teacher time and therefore affect read¬ 
ing, or how the school’s planning and problem-solving procedures are applied to the 
reading program. We accomplished this by means of personal interviews with school 
people. 

The other phase of our work in the 20 schools largely consisted of the administra¬ 
tion of a series of questionnaires. Using the information and understanding we 
gained in the three detailed studies, we constructed a group of interview forms and 
questionnaires to enable us to gather quantifiable information that would be compa- 
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rable across all the schools in our sample. We interviewed several staff members at 
each school, because we believe that people with different roles in the reading 
program—teachers as compared with principals, for instance—have different in¬ 
sights into how it worked. 

We received remarkably full cooperation from the administration, staff, and 
faculty of all 20 schools. All principals and all reading coordinators agreed to be 
interviewed or completed questionnaires. Of the 83 teachers who were responsible 
for 6th grade classes in 1974 and 1975, 81 completed questionnaires, even though 
many were teaching in other schools by the time we traced them, and one was 
teaching in another state. Our access to school personnel and records was consis¬ 
tently facilitated. 



Appendix B 


SIXTH GRADE READING ACHIEVEMENT AS A FUNCTION 
OF INITIAL ACHIEVEMENT AND BACKGROUND 


Variable 

Black 

Children 

Mexican 

American 

Children 

Fifth grade score 

.74 (15.32) 

.74 ( 

21.90) 

Male 

“.66 (- .51) 

1.41 ( 

1.56) 

Days absent 

-.19 (-2.98) 

-.03 ( 

-.91) 

Health factors cited 

2.98 ( 1.85) 

-.52 ( 

-.53) 

Family: mother, father 

-.56 (- .18) 

-1.65 ( 

-.84) 

mother only 

.43 ( .14) 

■ “.75 ( 

-.34) 

guardians 

-2.87 ( -.68) 

.26 ( 

.07) 

Father: professional, technical 

-5.78 (-1.38) 

5.03 ( 

1.39) 

white collar 

-7.83 (-2.07) 

.32 ( 

.10) 

craft worker 

-4.41 (-1.26) 

.66 ( 

.35) 

operative 

-10.26 (-2.94) 

.02 ( 

.01) 

laborer 

-6.69 (-2.12) 

1.45 ( 

.73) 

service worker 

-5.84 (-1.78) 

-.33 ( 

-.14) 

occupation blank 

-4.84 (-1.67) 

.33 ( 

.18) 

Mother: white collar 

-2.40 ( -.64) 

3.05 ( 

.90) 

service worker 

-4.27 (-1.15) 

1.92 ( 

.66) 

operative 

1.66 ( .40) 

-3.74 (- 

-1.58) 

housewife 

-1.06 ( -.33) 

-.41 ( 

.21) 

occupation blank 

-.58 ( -.18) 

“1.60 ( 

-.73) 

Age in months 

-.31 (-2.21) 

-.16 ( 

-1.79). 

Remedial reading pullout 

-10.21 (-5.28) 

-3.13 (- 

-1.85) 

Gifted program pullout 

2.08 ( .58) 

1.61 ( 

1.03) 

Constant 

81.13 ( 3.81) 

50.33 \ 

3.62) 

R 2 

.581 

.675 


Number of observations 

356 

394 



NOTES: Entries are regression coefficients, with t-statistics in parentheses. 
Excludes observations obtained from classrooms in which test exposure was 
reported or indicated by field research. 

Because of the varying quality of some of the background data, and because 
of the substantial multieollinearity in the sample,.it is unwise to interpret the 
size and direction of individual coefficients. These regression equations are 
simply designed to control for as much of the pre-6th-grade background of each 
child as possible; no further inference is intended. 
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Appendix C 

VARIABLES IN REGRESSION ANALYSIS OF READING 
ACHIEVEMENT FOR BLACK CHILDREN 


I. BACKGROUND VARIABLES 

6th grade score. Raw score on CTBS-Q2, given May 1975. 

5th grade score. Raw score on CTBS-R2, given May 1974. 

Sex. Dummy variable for male student. 

Attendance. Days absent during 1974-75. 

Health problems. Dummy variable for whether any "significant health factors” 
were noted on child’s record. 

Family status. Dummy variables for mother and father present; mother only 
present; guardians present, from child’s record. 

Father’s occupation. Dummy variables for selected occupations: professional, tech¬ 
nical, and kindred worker; white-collar worker; craft worker; operative; labor¬ 
er; service worker; father’s occupation missing. 

Mother’s occupation. Dummy variables for selected occupations: white-collar 
worker; operative; service worker; housewife; mother’s occupation missing. 

Age. Age in months. 

Remedial reading. Dummy variable for whether child was "pulled out” of regular 
classroom for supplementary instruction in reading by a teacher other than the 
regular teacher. 

Gifted program. Dummy variable for whether child was "pulled out” of the regular 
program to participate in special activities for gifted children. 


II. CLASSROOM VARIABLES (REPRODUCED FROM THE 
TEACHER QUESTIONNAIRE) 

Disruptions. In a typical month last year, how often did you have severe disruptions 
(fights, loud or boisterous play, vandalism) in your classroom? 


Never.1 

Once or twice.2 

3—5 times .3 

6—10 times, , ... . 4 

More than 10 times.5 


Number of parent visits. About how many of your students’ parents came into the 
classroom for a visit last year? 

Sense of efficacy. Product of: 

Please indicate whether you agree or disagree with each of the following 
statements about this district, this school, and about teaching. 
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Strongly 

Agree 

Agree 

Neither 
Agree Nor 
Disagree 

Disagree 

Strongly 

Disagree 

When it comes right down to it, 
a teacher really can’t do much— 






most of a student’s motivation 
and performance depends on his 
or her home environment 

1 

2 

3 

4 

5 

If I try really hard, I can get through 

to even the most difficult or un¬ 
motivated students 

5 

4 

3 

2 

1 


Variety of materials, with amount of training. Linear combination of: 

Please indicate the extent to which your school’s reading program affected 
the following aspects of your teaching practices. 


Your use of a greater variety of materials for different children at any one time: 


7 6 5.4 3 2 1 

I__I_I_1_I_I_1 

Great impact No impact 


About how many hours of training sessions (lectures or workshops) did you 
attend before you began implementing the reading program in your class¬ 
room? 


None. 1 

6 or fewer hours (consider 6 hours 

one work day). . , .... 2 

7 to 18 hours (consider 18 hours 

three work days). 3 

19 to 30 hours (consider 30 hours 

one work week) . 4 

More than 30 hours... 5 


Teacher adaptation of program. Product of: 

To what extent were teachers encouraged to adapt or modify the reading 
program on an individual classroom basis? 


7 6 5 4 3 2 1 

I-1_I_I_I_1__J 

To a great extent Not at all 
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Please indicate whether you agree or disagree with each of the following 
statments about this district, this school, and about teaching. 



Strongly 

Agree 

Agree 

Neither 
Agree Nor 
Disagree 

Strongly 
Disagree Disagree 

A teacher’s reading instructions 
was expected to conform closely 
to the school’s reading program 
guidelines; making one’s own 
modifications in reading instruc¬ 
tion was discouraged 

1 

2 

3 

4 5 


Teacher-to-teacher consultations. 

How much did you consult with other teachers on an informal basis about 
the reading program and your work with it? 


7 6 5 4 3 2 1 

I_I_I_I_I_I_J 

A great deal Not at all 



